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Why  GAO  Did  This  Study 

The  satellite-based  GPS  provides 
positioning,  navigation,  and  timing  data 
to  users  worldwide.  The  Air  Force  is 
modernizing  the  satellite,  ground 
controi,  and  user  equipment  segments 
to  enhance  GPS  performance. 

The  Senate  and  House  Armed 
Services  Committee  reports 
accompanying  bills  for  the  National 
Defense  Authorization  Act  for  Fiscal 
Year  2015  included  provisions  for  GAO 
to  review  the  status  of  OCX 
development  and  DOD’s  efforts  to  field 
M-code  signal  capability.  This  report 
addresses  (1 )  the  extent  to  which  DOD 
is  meeting  cost,  schedule,  and 
performance  requirements  for  OCX;  (2) 
the  progress  DOD  is  making  in 
deiivering  M-code  capabie  MGUE  by 
the  end  of  fiscai  year  2017;  and  (3)  the 
challenges  DOD  faces  in  synchronizing 
the  development  of  GPS  III,  OCX,  and 
MGUE  to  deploy  M-code. 

To  conduct  this  work,  GAO  analyzed 
program  documents  such  as 
acquisition  strategies;  reviewed 
oversight  reporting;  assessed 
constellation  reliability  metrics;  and 
interviewed  officiais  from  DOD 
programs  and  contractors. 

What  GAO  Recommends 

GAO  recommends  that  DOD  obtain  a 
more  robust  independent  assessment 
of  OCX  to  identify  and  resolve  root 
causes,  and  ensure  MGUE  design  is 
stable  to  inform  testing  and 
procurement  decisions.  DOD 
concurred  on  OCX  but  stated  that 
actions  taken  to  date  are  sufficient. 
DOD  partially  concurred  on  MGUE. 
GAO  believes  all  recommended 
actions  are  necessary  to  address 
systemic  problems. 

View  GAO-15-657.  For  more  information, 
contact  Cristina  Chapiain  at  (202)  512-4841  or 
chapiainc@gao.gov. 


What  GAO  Found 


The  Air  Force  has  experienced  significant  difficulties  developing  the  Global 
Positioning  System  (GPS)  next  generation  operational  control  system  (OCX)  and 
consistently  overstated  progress  to  the  Office  of  the  Secretary  of  Defense  (OSD) 
compared  to  advisory  independent  assessments  it  received.  It  needs  $1.1  billion 
and  4  years  more  than  planned  to  deliver  OCX  due  to  poor  acquisition  decisions 
and  a  slow  recognition  of  development  problems.  The  Air  Force  began  OCX 
development  in  2010  prior  to  completing  preliminary  development  reviews  in 
contrast  with  best  acquisition  practices.  It  accelerated  OCX  development  in  2012 
to  meet  optimistic  GPS  III  satellite  launch  timeframes  even  as  OCX  development 
problems  and  costs  grew,  and  then  paused  development  in  2013  to  address 
problems  and  resolve  what  it  believed  were  root  causes.  However,  as  the  figure 
below  shows,  OCX  cost  and  schedule  growth  have  persisted  due  in  part  to  a  high 
defect  rate,  which  may  result  from  systemic  issues.  Further,  unrealistic  cost  and 
schedule  estimates  limit  OSD  visibility  into  and  oversight  over  OCX  progress. 


OCX  Cost  and  Schedule  Changes  Since  Contract  Award  in  2010 
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Number  of  months  from  contract  award 
in  February  2010  to  block  1  completion 


The  Air  Force  is  implementing  the  military  GPS  user  equipment  (MGUE)  program 
to  develop  for  the  military  services  GPS  receiver  cards  capable  of  receiving  the 
military-code  (M-code)  signal — which  can  help  users  operate  in  jamming 
environments.  The  Air  Force  has  revised  MGUE’s  acquisition  strategy  several 
times  in  its  quest  to  develop  the  cards.  Even  so,  the  military  services  are  unlikely 
to  have  sufficient  knowledge  to  make  informed  procurement  decisions  starting  in 
fiscal  year  2018  because  operational  testing  that  provides  valuable  information 
about  MGUE  performance  will  not  be  complete  until  fiscal  year  2019. 


The  current  GPS  constellation  has  proven  to  be  much  more  reliable  than  the  Air 
Force  predicted  when  GAO  last  reported  on  it  in  2010.  Given  delays  to  OCX,  the 
Air  Force  has  prepared  contingency  plans  for  sustaining  the  GPS  constellation, 
but  these  plans  may  not  deliver  the  full  range  of  GPS  capability.  Initial  M-code 
capability  will  not  be  available  until  OCX  delivery  in  mid-201 9  at  the  earliest  and 
full  M-code  capability  is  likely  at  least  a  decade  away — once  the  services  are 
able  to  deploy  MGUE  receivers  in  sufficient  numbers.  Until  the  OCX  program 
trajectory  is  corrected,  additional  delays  to  it  may  likely  pose  significant  risks  to 
sustaining  the  GPS  constellation  and  delivering  GPS  capability. 
_ United  States  Government  Accountability  Office 
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Abbreviations 

ACER 

annual  GPS  enterprise  review 

CDR 

critical  design  review 

D3 

DAGR  Distributed  Device 
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defense  acquisition  board 

DAGR 

Defense  Advanced  GPS  Receiver 

DCMA 

Defense  Contract  Management  Agency 
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Department  of  Defense 
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Director,  Operational  Test  and  Evaluation 

FY 
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GPNTS 

GPS-based  Positioning,  Navigation,  and  Timing  Service 
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Global  Positioning  System 

lA 

information  assurance 
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Joint  Light  Tactical  Vehicle 

M-code 

military  code 

MGUE 

military  GPS  user  equipment 

MUE 

modernized  user  equipment 
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operational  control  system 
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next  generation  operational  control  system 

OSD 

Office  of  the  Secretary  of  Defense 

OTB 

over  target  baseline 
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positioning,  navigation,  and  timing 

USD  AT&L 

Under  Secretary  of  Defense  for  Acquisition,  Technology, 
and  Logistics 
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Washington,  DC  20548 


U.S.  GOVERNMENT  ACCOUNTABILITY  OFFICE 


September  9,  2015 
Congressional  Committees 

The  satellite-based  Global  Positioning  System  (GPS)  provides 
positioning,  navigation,  and  timing  (PNT)  data  to  users  worldwide. 

Besides  being  a  household  term  and  synonymous  with  navigation,  GPS  is 
an  essential  U.S.  national  security  asset  and  a  key  component  of 
economic  growth,  national  infrastructure,  and  transportation  safety.  The 
Department  of  Defense  (DOD) — specifically,  the  Air  Force — develops  and 
operates  the  GPS  system,  which  consists  of  three  segments:  space, 
which  comprises  a  constellation  of  PNT  satellites  orbiting  the  earth — 
currently,  approximately  40;  ground  control,  which  primarily  consists  of 
software  and  operates  the  satellites  as  well  as  monitors  and  corrects 
signal  data;  and  receivers,  which  help  civil  and  military  users  employ  GPS 
signals  to  determine  their  location.  For  about  the  past  8  years,  the  Air 
Force  has  been  in  the  process  of  modernizing  all  three  segments  to 
enhance  GPS  performance  and  security.  This  modernization  effort  is 
divided  into  three  major  programs:  GPS  III,  which  is  developing  a  new 
generation  of  satellites;  next  generation  operational  control  system 
(OCX),  which  will  replace  the  existing  ground  system  to  operate  most  of 
the  current  and  future  satellites;  and,  for  DOD  users,  military  GPS  user 
equipment  (MGUE),  which  will  provide  the  military  services  with  new 
receivers  that  can  receive  GPS  signals — particularly,  the  advanced 
military  code  (M-code)  signal — in  hostile  jamming  or  challenging 
environments. 

The  Senate  and  House  Armed  Services  committee  reports  accompanying 
bills  for  the  National  Defense  Authorization  Act  for  Fiscal  Year  2015 
included  provisions  for  GAO  to  review  the  status  of  OCX  development 
and  DOD’s  efforts  to  field  M-code  capability,  respectively.''  This  report 
assesses  the  (1)  extent  to  which  DOD  is  meeting  cost,  schedule,  and 
performance  requirements  for  OCX,  (2)  progress  DOD  has  made  in 
delivering  M-code  capable  MGUE  by  the  end  of  fiscal  year  201 7,  and  (3) 
challenges  DOD  faces  in  synchronizing  the  development  of  GPS  III 


^S.  Rpt  No.  113-176  at  242,  accompanying  S.  2410,  113*''  Cong.  (2014);  H.R.  Rep.  No. 
113-446  at  293,  accompanying  H.R.  4435,  113***  Cong.  (2014). 
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satellites,  OCX,  and  MODE  to  deploy  M-code.  To  conduct  our  work,  we 
analyzed  program  cost  and  schedule  baselines,  strategies,  schedules, 
software  development  plans,  earned  value  metrics,  and  other  documents. 
We  also  interviewed  officials  from  the  government  program  offices,  prime 
contractors.  Defense  Contract  Management  Agency  (DCMA),  the  Office 
of  the  Director,  Operational  Test  and  Evaluation  (DOT&E),  each  of  the 
military  services,  and  Air  Force  Space  Command,  and  analyzed  program 
and  management  briefings  and  GPS  constellation  reliability  parameters. 
We  examined  only  OCX  development  contract  costs  rather  than  full 
program  acquisition  costs  because  the  latter  includes  prior  expenditures 
on  technology  development  as  well  as  annual  management  and  GPS 
enterprise  integration  support  services.  We  did  not  include  an 
assessment  of  cost,  schedule,  or  performance  for  GPS  III  satellites  in  the 
scope  of  our  work.  In  addition,  we  did  not  review  MGUE  Increment  2 
because  that  development  effort  has  not  yet  reached  its  technology 
development  milestone.  Appendix  I  contains  a  more  detailed  description 
of  our  scope  and  methodology. 

We  conducted  this  performance  audit  from  August  2014  to  September 
2015  in  accordance  with  generally  accepted  government  auditing 
standards.  Those  standards  require  that  we  plan  and  perform  the  audit  to 
obtain  sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for 
our  findings  and  conclusions  based  on  our  audit  objectives.  We  believe 
that  the  evidence  obtained  provides  a  reasonable  basis  for  our  findings 
and  conclusions  based  on  our  audit  objectives. 


Background 


GPS  is  a  global  positioning,  navigation,  and  timing  system  consisting  of 
space,  ground  control,  and  user  equipment  segments  that  support  the 
broadcasts  of  military  and  civil  GPS  signals.  These  signals  each  include 
positioning  and  timing  information,  which  enables  users  with 
appropriately-equipped  GPS  receivers  to  determine  their  position, 
velocity,  and  time,  24  hours  a  day,  in  all  weather,  worldwide.  GPS  is  used 
by  all  branches  of  the  military  to  guide  troop  movements,  assist  with 
logistics  support  and  situational  awareness,  and  synchronize 
communications  networks.  In  addition,  weapon  systems,  including 
munitions,  are  guided  to  their  targets  by  GPS  signals,  and  GPS  is  used  to 
locate  military  personnel  in  distress.  Early  in  the  development  of  GPS,  its 
scope  was  expanded  to  include  complementary  civil  capabilities  such  as 
civil,  maritime,  and  land  navigation. 
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GPS  System  Description 


Space,  ground  control,  and  user  equipment  segments  are  needed  to  take 
full  advantage  of  GPS  capabilities.  The  GPS  space  segment  consists  of  a 
constellation  of  satellites  that  broadcast  encrypted  military  signals  and 
civil  signals.  In  recent  years,  because  numerous  satellites  have  lasted 
longer  than  anticipated,  the  constellation  has  grown  well  beyond  the 
minimum  requirement  of  24  satellites  to  approximately  40  satellites  of 
various  generations,  with  8  in  residual  status.  The  satellites  are  operated 
by  a  master  control  station  that  regularly  updates  navigation  signals  on 
the  satellites.  Using  these  navigation  signals,  GPS  military  and  civilian 
user  equipment  receivers  determine  a  user’s  location.  Figure  1  below 
illustrates  how  GPS  satellites,  ground  control,  and  user  equipment 
function  together  as  an  operational  system. 
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Figure  1 :  GPS  Operational  System 
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Source;  Copyright  ©Corel  Corp.  all  rights  reserved  (map);  Art  Explosiort  (images);  GAO  analysis.  |  GAO-15-657 


The  GPS  ground  control  segment  primarily  consists  of  software  deployed 
at  a  master  control  station  at  Schriever  Air  Force  Base,  Colorado,  and  at 
an  alternate  master  control  station  at  Vandenberg  Air  Force  Base, 
California.  The  ground  control  software  is  supported  by  6  Air  Force  and 
1 1  National  Geospatial-Intelligence  Agency  monitoring  stations  located 
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around  the  globe  along  with  four  ground  antennas  with  uplink  capabilities. 
Information  from  the  monitoring  stations  is  processed  at  the  Master 
Control  Station  to  determine  satellite  clock  and  orbit  status. 

The  GPS  user  equipment  segment  includes  military  and  civilian  GPS 
receivers.  These  receivers  determine  a  user’s  position  and  time  by 
calculating  the  distance  from  four  or  more  satellites  using  the  navigation 
signals  on  the  satellites  to  determine  its  location.  Military  GPS  receivers 
are  designed  to  utilize  the  encrypted  military  GPS  signals  that  are  only 
available  to  authorized  users,  including  military  and  allied  forces  and 
some  authorized  civil  agencies.  Civilian — including  commercial — 
receivers  use  the  civil  GPS  signal,  which  is  publicly  available  worldwide. 


Curr6nt  Mod6rnization  in  2000,  DOD  began  an  effort  to  modernize  the  space,  ground  control. 

Efforts  and  user  equipment  segments  of  GPS  to  enhance  the  system’s 

performance,  accuracy,  and  integrity.  To  that  end,  the  Air  Force  is  now  in 
the  process  of  developing  a  new  generation  of  GPS  III  satellites,  OCX, 
and  M-code  capable  MGUE  receivers. 

QPS  III  GPS  III  satellites  are  planned  to  supplement  and  eventually  replace  the 

constellation  of  GPS  satellites  now  in  orbit;  these  satellites  consist  of 
multiple  versions  or  generations  developed  and  launched  over  the  years. 
The  first  GPS  III  satellite  was  originally  expected  to  be  available  for 
launch  in  April  2014;  however,  due  to  development  problems  it  is  now 
expected  to  be  ready  for  launch  in  May  2017.  A  complete  GPS  III  satellite 
has  not  yet  been  tested,  and  the  program  is  now  rebaselining  its  cost 
estimates  as  a  result  of  the  schedule  delay  and  associated  increased 
costs. 

QQX  The  GPS  ground  control  segment  is  being  modernized  under  the  OCX 

program.  OCX  is  required  because  the  existing  GPS  control  system. 
Operational  Control  System  (OCS)  cannot  control — and  therefore 
enable — the  modernized  features  of  the  two  latest  generations  of  GPS 
satellites — IIR-M  and  IIF — currently  in  orbit.  The  Air  Force  plans  to 
develop  OCX  in  blocks,  with  each  block  delivering  upgrades  as  they 
become  available.  Block  0  is  intended  to  support  the  launch  and  initial 
testing  of  GPS  III  satellites;  block  1  is  designed  to  command  and  control 
the  GPS  II  and  III  satellites  and  basic  modernized  signals;  and  block  2  is 
to  enable  the  full  modernized  M-code  signal  and  support,  monitor,  and 
control  additional  navigation  signals.  OCX  block  1  is  needed  to 
incorporate  GPS  III  satellites  into  the  operational  constellation  to  sustain 
required  levels  of  GPS  signal  coverage,  because  the  legacy  OCS  system 
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cannot  support  the  GPS  III  satellites.  OCX  is  also  required  to  enable 
military  and  civil  use  of  modernized  GPS  signals.  In  particular,  the  military 
cannot  use  M-code  signals  for  enhanced  warfighting  until  OCX  block  1  is 
delivered. 

The  Air  Force  began  the  OCX  program  in  2007  with  a  technology 
development  phase,  referred  to  as  phase  A.  The  Air  Force  awarded 
phase  A  contracts,  for  approximately  $160  million  each,  to  Northrop 
Grumman  in  Redondo  Beach,  California,  and  Raytheon  Intelligence  and 
Information  Systems  in  Aurora,  Colorado  to  produce  competitive 
prototypes.  Following  the  competitive  down-select  between  these  firms’ 
prototypes,  the  Air  Force  awarded  the  development  contract  to  Raytheon 
in  February  2010.  This  $886  million  contract  covered  the  development  of 
OCX  blocks  1  and  2  (as  discussed  in  this  report,  block  0  was  added  as  a 
contract  modification  later),  with  an  option  to  begin  preliminary  work  on 
blocks  3  and  4  which  are  to  provide  additional  capabilities  to  support 
follow-on,  upgraded  versions  of  GPS  III  satellites. 

At  the  formal  start  of  development  (milestone  B),  the  Air  Force  estimated 
the  total  OCX  acquisition  costs — including  other  costs  such  as 
expenditures  on  technology  development  prior  to  2010  and  annual 
management  support  and  enterprise  integrator  services — at  $3.5  billion. 
According  to  Air  Force  documentation,  enterprise  integrator  services  are 
required  to  ensure  GPS  enterprise  coordination  among  the  ground, 
space,  and  user  equipment  segments.  These  costs  do  not  include  funding 
contributions  from  civilian  agencies  to  support  OCX;  according  to  Air 
Force  documentation,  the  Department  of  Transportation  and  other 
agencies  are  to  provide  resources  to  DOD  to  develop  and  operate  GPS 
civil  capabilities.  For  the  purpose  of  this  report,  OCX  development  costs 
refer  to  the  costs  of  the  development  contract  with  Raytheon. 

MQUE  Military  GPS  receivers  are  also  being  modernized  under  the  MGUE 

program.  The  Air  Force  was  directed  by  the  Assistant  Secretary  of 
Defense  for  Networks  and  Information  Integration  in  August  2006  to 
develop  M-code  capable  GPS  receiver  cards  to  meet  military  services’ 
needs.  In  January  201 1 ,  the  Ike  Skelton  National  Defense  Authorization 
Act  for  Fiscal  Year  201 1  directed  that  DOD  not  obligate  or  expend  funds 
to  procure  GPS  user  equipment  after  fiscal  year  2017  unless  that 
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equipment  is  capable  of  receiving  M-code.^  Figure  2  below  shows  an 
illustration  of  a  MGUE  receiver  card. 


Figure  2:  MGUE  Receiver  Card 


Source;  U.S- Air  Force.  |  GAO-15-657 


The  modernized  receiver  cards  are  to  provide  U.S.  forces  with  enhanced 
position,  navigation,  and  time  capabilities,  while  improving  resistance  to 
existing  and  emerging  threats,  such  as  jamming.  The  Air  Force  plans  to 
develop  MGUE  in  two  increments.  The  first  increment,  now  underway 
and  expected  to  cost  about  $1.7  billion,  utilizes  three  prime  contractors. 
All  three  contractors — L-3  Interstate  Electronics  Corporation,  Raytheon 
Space  and  Airborne  Systems,  and  Rockwell  Collins — are  developing 
receiver  cards  for  ground  environments.  Raytheon  and  Rockwell  Collins 
are  also  developing  combined  aviation/maritime  receiver  cards  for  use  in 
air  and  sea  environments.  The  Air  Force  plans  to  build  on  the  work 
conducted  in  Increment  1  to  develop  a  more  compact  receiver  card  in 
Increment  2  that  can  be  used  when  size,  weight,  and  power  need  to  be 
minimized  and  that  can  serve  as  an  “engine”  for  future  military  GPS 
receivers.  At  this  time,  the  Air  Force  is  also  exploring  the  possibility  of 
using  Increment  1  technologies  to  support  munitions,  handheld  devices, 
and  space  applications  that  it  previously  anticipated  supporting  with 
Increment  2. 


^Pub.  L.  No.  1 11-383  §  913,  which  provides  that  unless  a  waiver  is  obtained  none  of  the 
funds  authorized  to  be  appropriated  or  otherwise  made  available  under  the  act  or  any 
other  act  for  DOD  may  be  obligated  or  expended  to  purchase  GPS  user  equipment  during 
fiscal  years  after  fiscal  year  201 7  unless  the  equipment  is  capable  of  receiving  M-code. 
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The  Air  Force  initially  began  development  of  M-code  in  fiscal  year  2003.  It 
then  transitioned  that  work  to  the  modernized  user  equipment  (MUE) 
program  in  2006.  According  to  GPS  program  updates,  MUE  was  troubled 
by  issues  such  as  underestimation  of  software  complexity,  longer  than 
anticipated  software  development  time,  and  more  difficult  than  expected 
software  and  hardware  integration.  These  issues,  among  others,  resulted 
in  significant  cost  and  schedule  growth.  The  MUE  program  yielded 
aviation  and  ground  prototypes  of  M-code-capable  components,  cost 
about  $498  million,  and  spanned  about  7  years,  ending  in  2013. 

Concurrent  with  and  based  on  MUE  development  work,  the  Air  Force 
initiated  the  MGUE  program  in  201 1 .  While  DOD  policy  on  GPS  user 
equipment  and  procurement  in  2006  indicated  the  Air  Force  was  to 
develop  MGUE  to  production-ready  status,  the  Air  Force  currently  plans 
to  develop  MGUE  Increment  1  ground  and  aviation/maritime  receiver 
cards  to  the  point  of  production  representative  test  articles.^  It  then 
intends  for  the  MGUE  program  office  to  provide  funding  to  the  military 
services  so  that  they  can  acquire,  integrate,  and  operationally  test  the 
receiver  cards  on  service-specific  “lead  platforms.”  These  platforms  are 
expected  to  serve  as  pathfinders  for  the  military  services’  ground, 
aviation,  and  maritime  environments  and  are  currently  designated  by  the 
services  as  follows:  Army-Defense  Advanced  GPS  Receiver  (DAGR) 
Distributed  Device  (D3)  onboard  a  Stryker  ground  combat  vehicle;  Air 
Force-B-2  Spirit  aircraft;  Navy-DDG-51  Arleigh  Burke  Destroyer  hosted 
by  GPS-based  Positioning,  Navigation,  and  Timing  Service  (GPNTS);  and 
Marine  Corps-Joint  Light  Tactical  Vehicle  (JLTV).  Once  operational 
testing  is  complete,  the  military  services  will  then  be  responsible  for 
procuring  MGUE  Increment  1  for  their  weapons  systems. 

DOD  indicates  that  the  military  services  anticipate  procurement  of  about 
207,000  MGUE  Increment  1  receiver  cards.  Historically,  full  fielding  of 
user  equipment  has  lagged  behind  the  anticipated  availability  of  GPS 
satellite  and  ground  systems.  For  example,  in  2009  we  reported  that  the 
Air  Force  expected  GPS  satellite  and  ground  systems  to  be  available  in 
2013  to  transmit  and  process  the  M-code  signal,  but  that  fielding  of  M- 


^The  Defense  Acquisition  Guidebook  states  that  production  representative  systems 
include  any  system  accurately  representing  its  final  configuration  using  mature  and  stable 
hardware  and  software;  that  accurately  mirrors  the  production  configuration,  but  not 
produced  on  a  final  production  line  (although  production  tooling  may  account  for  some 
components).  Department  of  Defense,  Defense  Acquisition  Guidebook  (May  2013, 
updated  September  2013),  ch  9,  para.  9.3.2. 
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code  equipment  to  all  designated  military  users  would  not  be  complete 
until  2025/ 


Poor  Acquisition 
Decisions  and  a  Slow 
Recognition  of 
Development 
Problems  Led  to 
Significant  OCX  Cost 
Increases  and 
Schedule  Delays 


OCX  development  contract  costs  have  more  than  doubled  since  the 
contract  was  awarded  in  February  2010,  increasing  by  approximately 
$1.1  billion  to  $1.98  billion,  and  the  program’s  schedule  has  roughly 
doubled  over  estimates  at  contract  award.  The  Air  Force  awarded  the 
contract  to  begin  OCX  development  but  did  not  follow  key  acquisition 
practices  such  as  completing  a  preliminary  design  review  before 
development  start  as  called  for  by  best  practices.  In  addition,  key 
requirements,  particularly  for  cybersecurity,  were  not  well  understood  by 
the  Air  Force  and  contractor  at  the  time  of  contract  award.  The  contractor, 
Raytheon,  experienced  significant  software  development  challenges  from 
the  onset,  but  the  Air  Force  consistently  presented  optimistic 
assessments  of  OCX  progress  to  acquisition  overseers.  Figure  3  below 
shows  select  key  events  related  to  the  OCX  program  since  the 
development  contract  was  awarded  in  2010. 


^GAO,  Global  Positioning  System:  Significant  Challenges  In  Sustaining  and  Upgrading 
Widely  Used  Capabilities,  GAO-09-325  (Washington,  D.C.:  Apr.  30,  2009). 
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Figure  3:  Time  Line  of  Seiect  Key  Events  for  the  OCX  Program 
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Source:  GAO  analysis  of  DOO  information  |  GAO-15-657 


Further,  the  Air  Force  complicated  matters  by  accelerating  OCX 
development  to  better  synchronize  it  with  the  projected  completion  time 
lines  of  the  GPS  III  satellite  program,  but  this  resulted  in  disruptions  to  the 
OCX  development  effort.  As  Raytheon  continued  to  struggle  developing 
OCX,  the  program  office  paused  development  in  late  2013  to  fix  what  it 
believed  were  the  root  causes  of  the  development  issues,  and 
significantly  increased  the  program’s  cost  and  schedule  estimates. 
However,  progress  reports  to  DOD  acquisition  leadership  continued  to  be 
overly  optimistic  relative  to  the  reality  of  OCX  problems.  OCX  issues 
appear  to  be  persistent  and  systemic,  raising  doubts  whether  all  root 
causes  have  been  adequately  identified,  let  alone  addressed,  and 
whether  realistic  cost  and  schedule  estimates  have  been  developed. 
Figure  4  below  shows  how  OCX  costs  have  grown  and  the  schedule 
delayed  since  contract  award  in  February  2010. 
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Figure  4:  OCX  Cost  and  Schedule  Changes  Since  Contract  Award  in  2010 


Dollars  (in  millions) 
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Cost  estimate  at  completion 

Number  of  months  from  contract  award  in  February  2010  to  block  1  completion 
Source:  GAO  analysis  of  DOD  data.  |  GAO-15-657 


The  Air  Force  Did  Not 
Follow  Key  Acquisition 
Practices  When  It  Initiated 
OCX  Development 


The  Air  Force  awarded  the  OCX  development  contract  to  Raytheon  in 
February  2010  for  $886  million;  blocks  1  and  2  were  forecast  for 
completion  in  August  2015  and  March  2016,  respectively.  The 
development  contract  was  awarded  before  completing  a  milestone  B 
decision  formally  authorizing  the  start  of  development.  In  addition,  the 
program  did  not  complete  a  preliminary  design  review  (PDR)  as  called  for 
by  best  practices;  the  Air  Force  subsequently  acknowledged  in  the  OCX 
acquisition  strategy  approved  in  September  2012  that  the  contract  was 
awarded  earlier  than  normal.  Figure  5  below  depicts  key  events  in  DOD’s 
typical  acquisition  process  and  the  corresponding  knowledge  called  for  in 
GAO’s  prior  work  on  best  product  development  practices.  As  shown  in  the 
figure,  our  prior  work  has  identified  several  proven  management  practices 
that,  if  fully  implemented,  can  help  DOD  minimize  cost  overruns  by 
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ensuring  programs  are  established  after  matching  requirements  and 
resources.^ 


Figure  5:  DOD  Acquisition  Process  and  GAO  Knowiedge-Based  Acquisition  Process 
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Source:  DOD  Instruction  5000.02  and  GAO  Best  Practices.  |  GAO-15-657 


^GAO,  Best  Practices:  Better  Management  of  Technology  Development  Can  Improve 
Weapon  System  Outcomes,  GAO/NSIAD-99-162  (Washington,  D.C.:  July  30,  1999);  Best 
Practices:  Better  Matching  of  Needs  and  Resources  Will  Lead  to  Better  Weapon  System 
Outcomes,  GAO-01-288  (Washington,  D.C.:  Mar.  8,  2001);  Best  Practices:  Capturing 
Design  and  Manufacturing  Knowledge  Early  Improves  Acquisition  Outcomes, 

GAO-02-701  (Washington,  D.C.:  July  15,  2002);  Defense  Acquisitions:  A  Knowledge- 
Based  Funding  Approach  Could  Improve  Major  Weapon  System  Program  Outcomes, 
GAO-08-619  (Washington,  D.C.:  July  2,  2008);  Best  Practices:  DOD  Can  Achieve  Better 
Outcomes  by  Standardizing  the  Way  Manufacturing  Risks  Are  Managed,  GAO- 10-439 
(Washington,  D.C.:  Apr.  22,  2010). 
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The  Air  Force  did  not  conduct  a  milestone  B  review  prior  to  awarding  the 
development  contract,  missing  an  opportunity  to  ensure  the  program 
began  on  a  sound  foundation.  A  milestone  B  review  is  an  important  point 
in  a  program  where  requirements  and  resources — that  is,  technology, 
design,  time,  and  funding — should  be  properly  matched  to  make  sure  the 
program  can  be  executed  as  planned.  The  Under  Secretary  of  Defense 
for  Acquisition,  Technology,  and  Logistics  (USD  AT&L)  approved  the  Air 
Force’s  request  in  January  2010  to  award  the  OCX  development  contract 
prior  to  milestone  B.  The  Air  Force  and  USD  AT&L  decided  to  award  the 
contract  early  for  several  reasons.  First,  the  Air  Force  believed  that  the  3- 
year  competitive  technology  development  effort  from  2007-2010,  where 
two  contractors  (Northrop  Grumman  and  Raytheon)  worked  on  proof-of- 
concept  prototypes  for  OCX,  successfully  demonstrated  initial 
functionality  and  reduced  risk.  Second,  the  Air  Force  wanted  to  reduce 
the  costs  of  carrying  two  contractors  through  PDR.  Finally,  the  Air  Force 
and  USD  AT&L  believed  that  splitting  efforts  between  two  contractors  was 
slowing  progress  on  the  program,  and  decided  that  down-selecting  to  one 
contractor  prior  to  milestone  B  would  help  accelerate  OCX  completion. 
According  to  Raytheon  officials,  a  contributing  factor  was  the  need  to 
align  OCX  development  with  that  of  GPS  III  development,  the 
development  contract  for  which  was  awarded  21  months  earlier,  in  May 
2008. 

In  addition,  prior  to  development  start,  the  program  did  not  complete  a 
PDR — which  assesses  the  maturity  of  the  preliminary  design  and 
confirms  that  the  system  is  ready  to  proceed  into  detailed  design  with 
acceptable  risk.  As  GAO’s  best  practices  work  has  shown,  PDR  is  a 
critical  step  at  which  customer  needs  are  balanced  with  available 
resources.®  Programs  that  are  launched  prior  to  completing  a  preliminary 
design  tend  to  experience  more  problems  compared  to  programs  where 
launch  occurs  after  completing  a  preliminary  design.^  Figure  6  below 
depicts  how  programs  generally  have  better  outcomes  when  they  are 
launched  after  conducting  relatively  more  detailed  systems  engineering: 
particularly,  completing  a  preliminary  design  before  formally  launching. 
Programs  launched  after  only  completing  a  notional  design — a  general 
concept,  unconstrained  by  resources,  of  what  the  product  will  look  like 


®GAO,  Best  Practices:  Setting  Requirements  Differentiy  Couid  Reduce  Weapon  Systems’ 
Totai  Ownership  Costs,  GAO-03-57  (Washington,  D.C.:  Feb.  11, 2003). 

^GAO-01-288. 
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OCX  Requirements  Were  Not 
Well  Understood 


and  what  it  might  be  capable  of — often  experienced  problems  and 
needed  more  time  or  money  than  had  been  estimated  at  program  launch. 


Figure  6:  General  Depiction  of  Level  of  Systems  Engineering  Compared  to  Program 
Outcomes 
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cases  Notional  design 
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Preliminary  design 

Final  design 


Source;  GAO  analysts.  1  GAO-15-657 


OCX  program  office  and  Raytheon  officials  stated  that  certain  OCX 
requirements  were  not  well  understood  at  the  time  of  contract  award.  In 
particular,  Raytheon  officials  stated  that  the  company  did  not  understand 
the  extent  to  which  it  would  be  required  to  implement  Information 
Assurance  (lA)  requirements  until  as  late  as  2013,  3  years  after  beginning 
OCX  development.  The  purpose  of  lA,  also  referred  to  as  cybersecurity, 
is  to  ensure  that  DOD  systems  can  resist  and  continue  to  operate  during 
cyber-attacks  by  managing  risks  and  implementing  safeguards.  OCX  was 
required  to  be  compliant  with  the  DOD  directive  that  prescribes  lA 
policies,  responsibilities,  and  procedures,  among  other  things,  for  DOD 
information  technology  and  defense  components.®  In  light  of  increasing 
cyber  threats  and  given  that  satellite  ground  systems  are  the  most 
vulnerable  components  to  potential  attacks,  the  Air  Force  plans  for  OCX 
to  have  an  improved  lA  capability  over  the  current  GPS  control  system. 
According  to  program  and  Raytheon  officials,  OCX  is  going  to  be  one  of 
the  first  large-scale  programs  within  DOD  to  fully  implement  lA 
requirements.  In  addition,  Raytheon  officials  described  cybersecurity 
threats  as  continuously  evolving,  and  that  both  Raytheon  and  the  Air 
Force  have  had  to  adapt  their  interpretation  over  time  of  how  to  meet  lA 
requirements  on  OCX  development  to  address  changing  threats. 


®DOD  Instruction  8500.01,  Cybersecur/fy  (Mar.  14,  2014),  which  reissues  and  renames 
DOD  Directive  8500.01  E,  Information  Assurance  (Oct  24,  2002)  and  incorporates  and 
cancels  DOD  Instruction  8500.02,  Information  Assurance  (lA)  Implementation  (Feb.  6, 
2003). 
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Preliminary  Design  Review 
Held  More  Than  a  Year  after 
Contract  Award 


Program  office  and  Raytheon  officials  noted  that  past  DOD  acquisition 
programs  routinely  waived  some  of  the  lA  requirements  in  prior  programs, 
and  that  Raytheon  entered  OCX  development  with  the  expectation  that 
some  lA  requirements  would  be  waived  as  in  the  past.  According  to 
Raytheon  officials,  at  the  time  OCX  development  began,  neither  the  Air 
Force  nor  Raytheon  had  the  experience — and  therefore,  the  knowledge — 
of  developing  such  a  complex  lA-intensive  program.  Given  the 
importance  of  GPS  to  the  military  and  civil  communities  and  with  the 
increase  in  cybersecurity  threats,  the  Air  Force  did  not  waive  any  lA 
requirements  for  OCX.  Consequently,  Raytheon  found  that  it  had  greatly 
underestimated  the  cost  and  time  to  meet  these  requirements.  According 
to  program  officials,  most  of  the  requirements  issues  were  resolved  in 
early  2015. 

The  OCX  program  held  PDR  in  August  2011 — more  than  a  year  after 
contract  award  and  after  the  Air  Force  had  spent  over  $1  billion  on  the 
overall  OCX  program  since  2007  including  phase  A  technology 
development — with  the  Air  Force  affirming  that  the  OCX  architecture  and 
design  was  solid  and  that  the  program  was  ready  to  begin  formal  system 
development.  However,  at  PDR,  Raytheon  did  not  prepare — and  the  Air 
Force  did  not  assess — a  preliminary  design  for  the  entire  OCX  system 
because  Raytheon  followed  an  iterative  software  development  process 
unlike  the  traditional,  “waterfall”  software  development  process.  An 
iterative  approach  is  a  common  industry  practice  that  consists  of 
developing  software  in  a  series  of  iterations  and  blocks,  where  developers 
go  through  multiple  cycles  of  breaking  down  requirements,  and  designing, 
coding,  and  integrating  software.  By  contrast,  in  the  traditional  waterfall 
approach  the  system  is  fully  designed  before  coding  and  testing  follow  in 
a  linear  sequence.  In  theory,  an  iterative  approach  allows  for  a  balanced 
and  efficient  use  of  resources  when  developing  complex  systems. 

Smaller  iterative  development  cycles — as  opposed  to  a  lengthy,  linear, 
waterfall  development  process — allow  for  capabilities  to  more  easily  be 
added  incrementally,  lessons  learned  to  be  incorporated,  and  early 
integration  testing  to  be  conducted  to  minimize  cost  and  schedule  risk.  To 
achieve  this,  effective  systems  engineering — the  process  of  deriving  and 
allocating  requirements  for  individual  software  iterations — is  key.  The  Air 
Force,  acknowledging  Raytheon’s  use  of  an  iterative  approach  in  late 
2010,  tailored  its  preliminary  design  review  criteria — which  originally  were 
based  on  a  traditional,  waterfall  approach — to  only  review  the  two 
iterations  completed  by  Raytheon  at  that  point.  For  the  remaining  six 
iterations,  Raytheon  had  only  completed  the  initial  allocation  of 
requirements  at  a  high  level.  However,  as  Raytheon  encountered 
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problems  in  software  development,  it  began  deferring  difficult  work  to  later 
iterations. 

Raytheon  was  experiencing  difficulties  developing  OCX,  but  the  Air  Force 
presented  optimistic  assessments  of  progress  to  USD  AT&L.  By  October 
201 1 ,  an  independent  OSD-chartered  review  team — one  of  a  series  of 
independent  reviews — warned  of  severe  software  development  problems, 
particularly  with  mounting  deferred  work,  ineffective  integration  testing, 
and  overestimated  software  productivity  rates,  and  predicted  a  15  to  19 
month  delay  to  block  1  delivery.  According  to  an  official  with  USD  AT&L, 
these  independent  assessments  are  designed  to  identify  program  cost, 
schedule,  and  performance  risks  and  provide  feedback  to  the  program 
manager  and  the  milestone  decision  authority.  The  official  further  noted 
that  the  assessments  were  considered  helpful,  but  essentially  nonbinding 
advice  from  department  experts  to  the  program  manager.  The 
independent  team  followed  up  in  February  2012  for  a  more  detailed 
assessment  of  OCX  and  predicted  a  higher,  24-month  slip  to  block  1 . 
However,  at  the  annual  GPS  enterprise  review  (AGER)  with  USD  AT&L  in 
April  2012,  the  Air  Force  acknowledged  the  software  issues  but  projected 
a  shorter,  1 1 -month  delay  citing  various  corrective  actions  already  taken, 
including  greater  oversight.®  This  projection  proved  to  be  overly  optimistic. 
In  light  of  the  challenges  experienced  to  that  point,  at  the  April  2012 
AGER,  USD  AT&L  postponed  the  scheduled  milestone  B  decision  for 
OCX  and  directed  the  program  to  return  for  a  formal  Defense  Acquisition 
Board  (DAB)  review  within  120  days  and  to  report  on  progress.'"’ 


®USD  AT&L  established  the  AGER  process  in  November  2009  to  initiate  a  consolidated 
defense  acquisition  board  (DAB)  review  of  the  GPS  enterprise,  consisting  of  space, 
ground  control,  and  user  equipment  segments,  in  order  to  maintain  efficient  oversight  of 
the  GPS  enterprise  and  ensure  successful  synchronization  and  execution.  According  to 
USD  AT&L  documentation,  prior  to  the  AGER  process,  each  GPS  program,  as  with  other 
DOD  acquisition  programs,  had  milestone  decision  authority  (MDA)-level  reviews  each 
year  for  each  increment,  and  this  number  of  reviews  threatened  to  overwhelm  MDA 
manpower  resources  while  preventing  the  Office  of  the  Secretary  of  Defense  (OSD)  from 
obtaining  a  consolidated  picture  of  progress  in  achieving  the  overarching  GPS  capability. 
The  AGER  process  is  intended  to  accomplish  a  yearly  MDA-level  program  review  of  all 
GPS  segments,  and  is  to  serve  as  the  primary  decision  point  for  ail  program  milestones. 

1  n 

As  major  defense  acquisition  programs  go  through  each  phase  of  the  acquisition 
process,  they  are  reviewed  by  the  Defense  Acquisition  Board,  which  is  chaired  by  USD 
AT&L  and  includes  the  secretaries  of  the  military  departments  and  other  senior  leaders. 
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The  Air  Force  Accelerated 
OCX  Development  to 
Meet  Optimistic  GPS  III 
Satellite  Launch  Time 
Frames  Even  as  OCX 
Was  Experiencing 
Problems 


In  the  midst  of  mounting  OCX  problems,  the  Air  Force  disrupted  the 
ongoing  software  development  effort.  As  it  became  apparent  that  OCX 
block  1  would  not  be  ready  in  time  to  support  the  then-projected  launch 
time  frames  of  the  first  GPS  III  satellite  (August  2015  and  May  2014 
respectively),  USD  AT&L  directed  the  Air  Force  in  January  201 1  to 
separate  out  development  of  the  satellite  launch  and  initial  testing  portion 
of  the  block  1  software.  Dubbed  block  0,  this  subset  of  software  is  to 
contain  the  capabilities  needed  to  launch  and  test  the  initial  GPS  III 
satellites.  The  Air  Force  modified  the  OCX  contract  in  January  2012  to 
implement  this  change,  which  included  bringing  forward  lA  capabilities 
originally  scheduled  for  later  iterations.  Block  0  is  a  temporary  measure 
because  the  Air  Force  anticipated  that  some  GPS  III  satellites  would  need 
to  be  launched  before  the  entirety  of  block  1  was  to  be  developed.  Block 
0  does  not  allow  the  GPS  III  satellites  to  be  incorporated  into  the  overall 
GPS  constellation  and  used  by  the  military  or  civil  community — which 
requires  block  1  implementation.  Further,  this  decision  to  accelerate  the 
GPS  III  launch  and  test  capability  was  based  on  Air  Force  assessments 
of  GPS  III  progress  that  appeared  overly  optimistic,  as  GAO  found  at  the 
time.'’''  Nevertheless,  according  to  Raytheon  officials,  the  creation  of  block 
0  and  acceleration  of  some  software  capabilities  caused  it  to  have  to 
revamp  the  OCX  software  development  plan  2  years  into  development — 
not  in  accordance  with  best  practices  in  software  development,  which  call 
for  stabilizing  requirements  and  design  prior  to  coding."'^  Our  prior  work  in 
this  area  shows  that  too  many  changes  to  requirements  can  result  in 
additional,  sometimes  unmanageable  risk. 


Meanwhile,  Raytheon  continued  to  struggle  with  developing  OCX, 
specifically  with  implementing  lA  requirements.  The  contractor  was  overly 
optimistic  in  its  initial  estimates  of  the  work  associated  with  incorporating 
open  source  and  reused  software,  and,  according  to  the  Air  Force,  did  not 
appear  to  follow  lA  screening  or  software  assurance  processes  as 


^^GAO,  Global  Positioning  System:  Challenges  In  Sustaining  and  Upgrading  Capabilities 
Persist,  GAO-10-636,  (Washington,  D.C.:  Sept.  15,  2010)  In  this  report,  we  noted  that 
while  the  GPS  III  program  (then  called  GPS  IMA)  had  adopted  practices  that  should  enable 
it  to  deliver  in  a  quicker  time  frame  than  the  prior,  troubled,  GPS  IIP  program,  the  inherent 
complexities  associated  with  the  work  yet  to  be  completed  likely  made  it  difficult  for  the 
GPS  III  program  to  meet  an  ambitious  schedule  that  was  about  3  years  shorter  than  the 
IIP  program. 

^^GAO,  Defense  Acquisitions:  Stronger  Management  Practices  Are  Needed  to  Improve 
DOD’s  Software-Intensive  Weapon  Acquisitions,  GAO-04-393  (Washington,  D.C.:  Mar.  1, 
2004). 
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required,  for  example,  incorporating  open  source  software  without 
ensuring  that  it  was  lA-compliant.  The  Air  Force  stated  that  it  was  not 
aware  that  Raytheon’s  software  assurance  processes  were  not  in 
compliance  with  the  OCX  software  development  plan  until  it  performed  an 
audit  of  Raytheon’s  secure  coding  process  in  August  2012.  According  to 
Raytheon  officials,  however,  the  development  contract  did  not  specifically 
address  the  extent  that  open  source  software  had  to  be  scanned  for  lA 
compliance.  Nevertheless,  this  led  to  significant  rework  and  added  cost  to 
remediate  the  security  vulnerabilities  and  meet  lA  standards.  In  addition, 
Raytheon’s  systems  engineering  was  incomplete,  resulting  in  an  inability 
to  build  code  as  planned  and  work  being  consistently  deferred  to  later 
iterations.  The  Air  Force  noted  that  it  became  aware  of  these  systems 
engineering  issues  early  in  development  and  took  some  corrective 
actions  such  as  defining  the  completion  of  certain  components;  however, 
these  actions  were  focused  only  on  high-priority  OCX  components. 

Nevertheless,  in  July  2012,  the  Air  Force  presented  a  positive  progress 
report  at  the  mandated  DAB  review,  stating  essentially  that  all  technical 
issues  had  been  identified  and  mitigated,  the  design  fully  validated,  and 
an  executable  and  realistic  schedule  put  in  place  after  a  thorough  internal 
examination  of  the  program.  While  the  program  stated  that  sufficient 
margin  had  been  incorporated  into  the  cost  and  schedule  estimates,  the 
program’s  estimates  for  block  1  delivery — between  February  and  October 
2016 — were  still  optimistic  compared  to  the  time  frames  (November  2016 
to  March  2017)  forecast  by  the  independent  team  in  October  2011.  The 
Air  Force  formally  completed  milestone  B  for  OCX  in  November  2012 — 
more  than  2  years  after  contract  award.  At  this  point,  the  Air  Force 
developed  its  first  detailed  cost  estimate  for  the  program  as  required  by 
DOD  policy,  forecasting  that  OCX  development  would  cost  approximately 
$1.6  billion,  which,  according  to  the  Air  Force,  was  its  first,  formal 
realization  of  the  magnitude  to  which  the  contract  was  initially  underbid. 


The  Air  Force  Paused 
OCX  Development  amid 
Ongoing  Problems  and  to 
Identify  and  Resolve 
Problems,  but  Continued 
to  Present  Optimistic 
Assessments  to 
Acquisition  Overseers 


According  to  the  OCX  program  office  and  independent  OSD-chartered 
reviews,  Raytheon’s  incomplete  systems  engineering  led  to  continuous 
rework  and  deferred  requirements  to  later  iterations.  In  addition,  the  Air 
Force  made  significant  changes  to  certain  requirements,  particularly  with 
updating  the  specifications  for  OCX’s  connections  to  other  government 
systems,  and  M-code  signal  requirements.  As  a  result,  Raytheon  officials 
estimated  that  nearly  two-thirds  of  the  requirements  baseline  as  of  PDR 
had  changed  by  mid-2012.  In  March  2013,  an  independent  OSD-led  team 
praised  the  program  for  the  corrective  steps  taken,  but  pointed  to  a 
rapidly  deteriorating  delivery  schedule  as  a  result  of  software 
development  taking  much  longer  than  planned  and  a  high  defect  rate. 
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among  other  factors.  The  independent  team  projected  block  1 
completion  between  February  and  July  2018,  a  schedule  slip  of  16  to  21 
months  over  the  estimate  at  milestone  B.  DOD’s  contract  performance 
reviewer,  Defense  Contract  Management  Agency  (DCMA),  highlighted 
major  technical  difficulties  in  OCX  development  in  its  monthly  analyses 
from  the  June  to  October  2013  time  frame,  including  missed  software 
iteration  milestones,  concurrent  systems  engineering  and  software 
development,  and  a  high  defect  rate,  among  other  issues.  In  addition, 
DCMA  reported  as  early  as  August  2013  that  the  Air  Force  and  Raytheon 
were  going  to  soon  begin  an  over  target  baseline  (0TB)  process  where 
DOD  determines  that  contract  budgets  are  unrealistic  and  formally 
increases  the  program’s  budget.  An  0TB  is  intended  to  allow  for  more 
realistic  budget  and  work  estimates  and,  therefore,  more  meaningful 
performance  measurement  against  the  updated  budget. 

However,  at  the  September  2013  AGER,  the  Air  Force  again  presented 
an  optimistic  assessment  of  the  program  to  USD  AT&L,  stating  that  OCX 
was  on  track  to  meet  milestone  B  cost  and  schedule  estimates,  lA 
challenges  had  been  identified,  and  key  metrics  had  been  established  to 
ensure  progress.  The  program  projected  just  a  2-month  delay  to  block  1 
(to  December  2016). 

Meanwhile,  the  Air  Force  noted  that  an  excessive  amount  of  rework  was 
occurring  at  the  time,  and  it  directed  Raytheon  in  November  201 3  to 
pause  development  and  complete  a  greater  level  of  systems  engineering 
for  block  1 .  In  December  the  Air  Force  and  Raytheon  began  an  0TB 
process  to  identify  root  causes  and  corrective  actions  and  establish 
revised  cost  and  schedule  goals  for  OCX,  and  notified  USD  AT&L  that 
block  1  delivery  was  likely  to  slip  by  approximately  9  months,  to 
September  2017. 

The  0TB  concluded  in  June  2014,  and  identified  root  causes,  many  of 
which  were  similar  to  those  identified  at  previous  AGERs  and 
independent  assessments:  incomplete  systems  engineering,  inadequate 
process  discipline,  and  lA  implementation  difficulties  due  to  complexity.  At 
this  point,  the  Air  Force’s  estimate  of  the  program’s  development  cost  had 


1 

The  OCX  Software  Development  Plan  defines  a  defect  as  a  finding  that,  if  not  changed, 
would  result  in  a  discrepancy  report  (DR)  against  the  code  in  a  later  phase  of  the 
development.  DRs  are  logged  in  a  tracking  system  to  ensure  corrective  actions  are  taken 
and  recorded  appropriately. 
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grown  to  $1.7  billion — about  6  percent  more  than  the  milestone  B 
estimate  and  nearly  twice  the  initial  contract  estimate  in  2010.  The  Air 
Force  also  added  a  total  of  2  years  to  the  program  over  its  estimate  at  the 
September  2013  AGER  with  block  1  delivery  moved  to  November  2018. 

Despite  the  ongoing  problems  with  OCX  development,  the  Air  Force, 
faced  with  the  statutory  deadline  to  deliver  M-code  capable  receivers — 
and  consequently,  OCX  support  for  M-code — by  fiscal  year  2018,  added 
some  uncertainty  to  the  OCX  development  plan.  The  Air  Force  agreed 
with  Raytheon  during  the  0TB  to  concurrently  develop  a  portion  of  block 
2  M-code  capability  during  the  block  1  effort;  however,  the  Air  Force 
noted  in  technical  comments  on  a  draft  of  this  report  that  Raytheon  is 
under  no  contractual  obligation  to  do  so.  Complicating  matters,  the  Air 
Force  made  significant  updates  to  M-code  signal  requirements  in  2014, 
contributing  to  additional  OCX  requirements  changes.  At  the  time  of 
contract  award  in  2010,  the  Air  Force  had  only  developed  a  preliminary 
M-code  specification,  and  it  took  time  for  the  Air  Force  to  fully  mature  M- 
code  requirements  and  specify  the  functionality  required  of  OCX. 
According  to  Raytheon  officials,  this  had  a  significant  effect  on  the  lA 
requirements  and  design  of  certain  components  of  OCX. 


The  OCX  Contractor  Has 
Continued  to  Struggle  with 
OCX  Development  and  a 
Realistic,  Executable 
Schedule  Is  Not  Yet 
Evident 


The  OCX  program’s  path  forward  following  the  conclusion  of  the  0TB 
process  in  June  2014  depended  on  resolving  certain  problems  in  order  to 
meet  revised  cost  and  schedule  targets.  In  particular,  the  Air  Force 
assumed  that  the  contractor  would  (1)  resolve  software  defects  quickly, 
(2)  bring  greater  discipline  to  software  development  processes,  and  (3) 
achieve  higher  software  productivity  rates  than  previously  demonstrated. 
However,  latest  available  data  as  of  May  2015  shows  that  the  contractor 
has  not  yet  resolved  these  issues,  increasing  the  risk  that  additional  OCX 
cost  and  schedule  growth  is  likely. 


Defect  Rate  Remains  High  and 
Root  Causes  May  Not  Be  Fully 
Identified 


A  key  expectation  underpinning  the  post-OTB  baseline  was  that 
Raytheon  would  be  able  to  sharply  reduce  the  number  of  new  defects  and 
resolve  them  within  6  months.  However,  the  opposite  took  place. 

Raytheon  uncovered  defects  at  a  faster  pace  than  it  could  resolve  them, 
causing  further  delays.  By  October  2014,  just  4  months  after  the  0TB, 
DCMA  reported  that  the  defect  resolution  rate  was  unsustainable, 
needing  a  continual  increase  in  cost  and  schedule,  with  the  trend  likely  to 
continue.  In  addition,  DCMA  found  that  the  majority  of  defects  were 
identified  during  the  later  phases  of  software  development  at  which  point 
it  is  harder  and  much  more  expensive  to  resolve  than  if  found  earlier. 
Latest  available  DCMA  reports,  as  of  May  2015,  showed  that  Raytheon 
had  yet  to  bring  the  defect  rate  within  planned  levels.  Raytheon  officials 
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noted  in  early  July  2015  that  the  defect  discovery  rate  and  backlog  have 
been  greatly  reduced  as  OCX  prepares  to  begin  testing  on  iteration  1 .5. 
However,  data  on  the  number  of  defects  open  and  resolved  show  that  the 
defect  backlog  as  of  July  2015  was  still  more  than  three  times  that 
predicted  in  February  2015.  In  addition,  DCMA  has  pointed  out  that  the 
test  activities  will  likely  identify  new  defects,  potentially  increasing  the 
backlog  further.  DCMA  further  noted  that  the  current  defect  backlog 
consists  of  more  complex  and  difficult  defects,  which  will  require 
considerable  effort  to  close  compared  to  those  resolved  so  far. 

The  persistently  high  defect  rate  for  CCX  may  be  a  result  of  as-yet 
unidentified  systemic  issues.  An  Cctober  2014  independent  review  noted 
that,  given  the  difficulty  in  resolving  defects  as  planned,  systemic  issues 
may  remain  but  that  neither  the  CCX  program  office  nor  Raytheon  had 
conducted  detailed  engineering  assessments  to  determine  if  there  were 
any  systemic  issues.  As  GAC’s  prior  work  has  shown,  effective  defect 
management  requires  a  realistic  schedule  in  that  it  takes  time  to  be  able 
to  fully  identify,  analyze,  prioritize,  and  track  defects.""^  Without  investing 
the  time  and  resources  to  conduct  detailed  engineering  assessments,  the 
program  cannot  know  if  any  systemic  issues  are  causing  the  persistently 
high  software  defect  rate. 


Pr0C6SS6S  R6inain  The  high  defect  rate  is  a  symptom  of  continued  struggles  with  the  root 

Undisciplined  cause  of  undisciplined  processes  at  Raytheon,  noted  to-date  by  the  CCX 

program  office,  independent  CSD-chartered  review  teams,  and  DCMA. 
For  example,  Raytheon  has  had  difficulty  establishing  consistent  software 
development  environments — the  computer  infrastructure  including 
hardware,  operating  systems,  and  databases — across  the  CCX  program. 
Developers  built  each  of  the  environments  with  different  hardware  and 
operating  system  versions  and  settings.  According  to  Raytheon  officials, 
this  was  partly  because  the  Air  Force’s  requirements  called  for  multiple 
tailored  environments  to  save  on  hardware  costs,  but  that  this  drove 
complexity  and  posed  technical  difficulties  to  Raytheon.  This  meant  that 
the  contractor  could  not  deploy  a  given  software  build  onto  all  the 
environments  without  a  lot  of  rework — and  consequently,  time  and 
expense.  Following  the  June  2014  CTB,  the  Air  Force  and  Raytheon 
reported  that  they  had  taken  steps  to  correct  this  issue,  such  as  using 


^■’^GAO,  Geostationary  Weather  Satellites:  Launch  Date  Nears,  but  Remaining  Schedule 
Risks  Need  to  Be  Addressed,  GAO-15-60,  (Washington,  D.C.:  Dec.  16,  2014). 
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Software  Productivity 
Assumptions  Are  Optimistic 


automated  tools  to  ensure  consistency.  However,  latest  available  data,  as 
of  May  2015,  showed  that  inconsistent  environments  were  still 
contributing  to  a  high  defect  rate  and  consequent  cost  growth  and  delays. 
Raytheon  noted  that  it  had  greatly  reduced  the  amount  of  time  required  to 
deploy  software  builds  onto  a  given  environment  to  3  days  as  of  July 
2015;  however,  this  was  still  well  short  of  the  1-day  limit  called  for  by  the 
October  2014  independent  review. 

Another  example  of  undisciplined  processes  at  Raytheon  is  its  peer 
review  process.  DCMA  reported  in  a  November  2014  audit  that  peer 
reviews — shown  by  our  prior  work  to  be  a  crucial  quality  assurance 
component  of  software  development — were  inconsistent  and  less 
effective  than  planned.''^  The  OCX  software  development  plan  identifies 
peer  reviews  as  an  integral  part  of  the  software  development  process,  the 
main  purpose  of  which  is  to  identify  and  report  defects  in  software 
artifacts — work  products  such  as  code,  software  class  libraries,  and 
design  models — as  early  as  possible  in  the  development  life  cycle. 
Effective  peer  reviews  depend,  in  part,  on  clear  instructions,  compliance, 
and  standards,  as  well  as  consistent  processes.  DCMA  reported 
Raytheon  having  problems  with  all  of  these  items.  In  its  audit,  DCMA 
noted  that  the  work  instructions  were  often  unclear,  contradictory,  or 
contained  loopholes;  many  peer  reviews  were  non-compliant  with  the 
work  instructions;  and  a  high  percentage  of  reviews  were  held  “virtually” 
where  users  review  artifacts  independently  instead  of  during  a  formal,  in- 
person  meeting  as  presumed  in  the  OCX  software  development  plan. 
DCMA  concluded  that  the  quality  of  the  peer  review  process  was 
questionable  because  peer-reviewed  software  artifacts  resulted  in  an 
abnormally  high  number  of  defects  being  discovered  later.  According  to 
Raytheon  officials,  Raytheon  has  incorporated  changes  to  its  peer  review 
instructions  in  its  software  development  plan,  which  is  awaiting  Air  Force 
approval  as  of  July  2015  and  will  govern  future  software  development. 

The  Air  Force  and  Raytheon  also  assumed  overly  optimistic  software 
productivity  rates — considering  Raytheon’s  track  record  on  OCX — when 
developing  the  post-OTB  schedule.  For  example,  although  the  post-OTB 
baseline  added  nearly  2  years  to  the  prior  plan,  both  the  Air  Force  and 
Raytheon  assumed  productivity  rates  for  iterations  1 .6  and  1 .7  that  were 
approximately  two-thirds  higher  than  the  rate  achieved  for  iteration  1 .5, 
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USD  AT&L  May  Not  Have 
Adequate  Insight  into  OCX 
Progress 


which  was  the  most  complex  software  effort  undertaken  to  that  point.  As 
of  the  latest  available  data,  from  February  2015,  Raytheon’s  productivity 
rate  had  increased  above  the  rate  achieved  for  1 .5  but  was  roughly  only 
half  the  forecast  increase  because  of  continued  difficulties,  increasing  risk 
that  OCX  delivery  will  be  further  delayed.  According  to  DCMA,  the 
program  has  had  a  history  of  being  overly  optimistic  with  forecasting 
schedules,  for  example,  using  data  from  Raytheon’s  prior  experience  on 
other  programs  that  have  consistently  proven  inaccurate  when  applied  to 
OCX,  and  assuming  efficiencies  based  on  learning  curves  which  have  not 
materialized  due  to  staff  turnover,  process  changes,  and  poor  retention  of 
lessons  learned  from  past  development  difficulties.  This  tendency  of 
overestimating  software  productivity  rates  was  noted  as  far  back  as 
October  201 1  as  part  of  an  independent  OSD-led  assessment  that 
highlighted  several  OCX  development  problems.  Raytheon  noted  that 
corrective  actions  were  taken  as  part  of  the  0TB,  such  as  revalidating 
and  completing  systems  engineering,  establishing  a  common 
environment  infrastructure,  and  validating  lA  implementation  during  block 
0  tests  will  result  in  software  productivity  improvement  for  iterations  1 .6 
and  beyond  beginning  in  2016. 

USD  AT&L  may  not  have  adequate  insight  into  the  full  extent  of  OCX 
development  problems  given  the  Air  Force’s  consistently  optimistic 
assessments  of  the  program’s  progress.  The  October  2014  advisory 
independent  review  estimated  that  block  1  was  likely  to  take 
approximately  2  years  longer  than  the  June  2014  0TB  estimate,  putting 
probable  block  1  delivery  around  November  2020.  DOT&E  also 
expressed  concerns  about  OCX  delays  in  a  memorandum  sent  to  USD 
AT&L  in  early  November  2014,  citing  the  negative  effect  of  those  delays 
on  the  Air  Force’s  ability  to  deliver  overall  GPS  capability.  In  addition, 
DOT&E  called  the  Air  Force’s  schedules  for  the  overall  GPS  enterprise, 
including  that  of  OCX,  “inaccurate,  implausible,  and  incoherent”  given 
OCX  development  difficulties  to  date. 

Nevertheless,  at  the  ensuing  fiscal  year  2015  AGER  that  was  held  in 
November  2014,  the  Air  Force  acknowledged  additional  development 
difficulties  but  noted  that  OCX  systems  engineering  was  improving  and 
would  lead  to  better  software  development  in  the  future.  At  that  time,  the 
OCX  program  office  forecast  block  1  completion  in  August  2019 — a  delay 
of  9  months  over  the  0TB  estimate,  but  optimistic  compared  to  the 
independent  review’s  assessment.  One  month  later,  in  December  2014, 
the  Air  Force  presented  its  official  update  for  the  program  and  forecast  a 
block  1  completion  date  of  July  2019.  It  also  estimated  that  contract  costs 
had  grown  to  nearly  $2  billion — representing  cost  growth  of  16  percent  in 
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just  5  months  over  the  0TB  estimate,  and  more  than  120  percent  over 
initial  contract  estimates  in  2010. 

Following  the  November  2014  AGER,  in  December  2014,  USDAT&L 
expressed  concern  about  the  continued  deterioration  in  the  cost 
performance  of  OCX  and  stated  that  the  program’s  trajectory  must  be 
corrected.  USD  AT&L  directed  the  Air  Force  to  provide  by  January  2015  a 
“deep  dive”  program  review  of  OCX  focused  on  the  status  of  and  ways  to 
improve  program  execution.  At  this  review,  which  occurred  in  February 
2015,  the  Air  Force  acknowledged  the  program’s  volatile  cost  and 
schedule  history  and  that  the  contractor’s  schedules  were  aggressive 
relative  to  the  risk  and  amount  of  work  remaining.  However,  the  Air  Force 
also  highlighted  signs  that  program  execution  was  stabilizing  and  noted 
that  it  had  reduced  program  risk  to  medium-low  by  including  an  additional 
$331  million  and  3  months  above  contractor  estimates.  To  address  USD 
AT&L’s  directive  to  detail  OCX’s  path  forward  and  key  decision  points 
should  OCX  continue  to  sustain  cost  and  schedule  growth,  the  Air  Force 
established  a  process  whereby  it  would  closely  monitor  key  software 
development  events  roughly  every  6  months  beginning  in  July  2015  and 
report  any  deviations  from  cost  performance  to  the  Air  Force’s  senior 
acquisition  executive  as  well  as  to  USD  AT&L  as  needed. 

While  the  closer  monitoring  of  key  software  development  events  provides 
USD  AT&L  with  opportunities  to  spot  cost  growth  on  a  more  timely  basis 
than  at  the  annual  AGERs,  there  is  little  reason  to  believe  that  OCX 
systemic  problems  have  been  adequately  addressed.  Notwithstanding  the 
Air  Force’s  optimistic  report  to  USD  AT&L  in  February  2015,  additional 
OCX  cost  growth  is  quite  likely.  First,  our  analysis  of  detailed  earned 
value  data  from  November  2013  through  November  2014  showed  that  the 
program  office  significantly  underestimated  the  anticipated  cost  of 
resolving  the  risks  that  could  affect  OCX  development  and  may  have  also 
significantly  underestimated  the  extent  of  risks.  Second,  DCMA’s 
reporting  of  earned  value  performance  for  the  program  shows  a  sustained 
deterioration  in  program  cost  and  schedule  performance  immediately 
following  the  0TB — where  cost  and  schedule  variances  were  reset  to 
zero  in  order  to  begin  measuring  performance  against  the  new  baseline. 
Cost  and  schedule  variances  measure  the  differences  in  expectations 
between  the  value  of  work  accomplished  in  a  given  period  with  the  value 
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of  the  work  expected  in  that  period.''®  Negative  variances  indicate  that  the 
program  is  either  overrunning  cost  or  performing  less  work  than  planned; 
conversely,  positive  variances  indicate  the  program  is  either  underrunning 
cost  or  performing  more  work  than  planned.  Figure  7  below  shows  the 
cumulative  cost  and  schedule  variances  from  October  2013  through  last 
available  data  as  of  May  2015,  and  how  the  OCX  program  has  been  both 
overrunning  cost  and  performing  less  work  than  planned  immediately 
after  the  0TB  concluded.  While  the  0TB  process  completed  in  June 
2014,  the  program  formally  reset  the  cost  and  schedule  baselines  at  the 
end  of  July. 


Figure  7:  OCX  Cost  and  Schedule  Variance  since  October  2013 
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Source:  GAO  analysis  of  OOO  data.  |  GAO-15-657 

Further,  DCMA’s  June  2015  analysis  forecast  that  OCX  costs  were  likely 
to  increase  to  $2.15  billion  based  upon,  among  other  things,  the  higher 
than  expected  defect  rate  and  poor  comprehension  of  requirements  by 
Raytheon.  From  November  2014  through  last  available  data  as  of  May 


^®GAO,  Cost  Estimating  and  Assessment  Guide:  Best  Practices  for  Deveioping  and 
Managing  Capitai  Program  Costs,  GAO-09-3SP,  (Washington,  D.C.:  March  2009). 
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2015,  Raytheon  depended  on  shifting  between  approximately  100  and 
180  additional  staff  than  planned  to  resolve  defects,  slowing  work  on  later 
iterations.  According  to  DCMA,  Raytheon  will  likely  need  even  more  staff 
in  the  future  to  maintain  schedule,  and,  consequently,  incur  additional 
cost  growth.  The  Air  Force’s  current  schedule  estimates,  forecasting 
block  1  completion  in  July  2019,  are  still  optimistic  by  at  least  a  year 
compared  to  the  October  2014  independent  team’s  assessment  that 
block  1  will  most  likely  be  delivered  in  November  2020.  Figure  8  below 
summarizes  the  differences  in  Air  Force  estimates  for  the  total  number  of 
months  to  complete  block  1  and  those  predicted  by  independent  reviews. 


Figure  8:  Comparison  of  Air  Force  and  Independent  Review  Estimates  of  Total 
Number  of  Months  to  Block  1  Completion 
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In  addition,  our  prior  work  on  space  acquisitions  has  shown  that 
unrealistic  estimates  of  the  achievability  of  planned  schedules,  among 
other  things,  directly  contributed  to  unrealistic  cost  estimates  and, 
consequently,  distorted  management  decisions,  increased  risks  to 
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mission  success,  and  virtually  guaranteed  program  delays.''^  By  contrast, 
senior  leaders  in  successful  organizations  in  the  commercial  sector 
actively  encouraged  program  managers  to  share  bad  news  about  their 
programs.  These  organizations  took  pains  to  ensure  program  estimates 
are  complete  and  accurate.  Based  on  the  persistently  high  software 
defect  rate,  continued  undisciplined  processes,  lower  than  expected 
software  productivity  rates,  the  downward  trend  in  cost  and  schedule 
performance  following  the  0TB,  and  lack  of  realism  in  Air  Force 
depictions  of  OCX  progress,  root  causes  do  not  appear  to  have  been 
adequately  addressed  or  perhaps  even  fully  identified.  Until  all  root 
causes  of  OCX  problems  are  fixed,  the  program  is  likely  to  continue  to 
struggle  to  achieve  desired  outcomes. 

Lack  of  Knowledge 
about  MGUE  Likely  to 
Hinder  Military 
Services’  Ability  to 
Make  Procurement 
Decisions 

The  Air  Force  has  revised  the  MGUE  acquisition  strategy  several  times  in 
attempts  to  develop  M-code  capability.  Even  so,  the  military  services  are 
unlikely  to  have  sufficient  knowledge  about  MGUE  design  and 
performance  to  make  informed  procurement  decisions  starting  in  fiscal 
year  201 8  because  it  is  uncertain  whether  an  important  design  review  will 
be  conducted  prior  to  that  time  and  because  operational  testing  will  still 
be  underway. 

MGUE’s  Acquisition 
Strategy  May  Limit  Insight 
into  MGUE  Design 

The  Air  Force  has  revised  the  MGUE  acquisition  strategy  several  times 
as  it  pursued  the  program’s  development.  The  latest  strategy  of  record 
eliminated  a  key  design  assessment,  the  critical  design  review  (CDR), 
and  it  is  uncertain  whether  a  recent  revision  to  that  strategy  will  include  a 
CDR.  As  a  result,  the  military  services  may  face  a  knowledge  gap  about 
MGUE’s  design  stability  and  maturity.  As  we  have  noted  in  our  prior  work, 
positive  acquisition  outcomes  typically  require  the  use  of  a  knowledge- 
based  approach  to  product  development  that  demonstrates  high  levels  of 
knowledge  before  significant  commitments  are  made.''®  We  determined 
that,  in  keeping  with  that  approach,  knowledge  gained  through  a  CDR 
helps  supplant  risk  over  time  by  ensuring  that  a  product’s  design  will  meet 

^^GAO,  Space  Acquisitions:  DOD  Needs  to  Take  More  Action  to  Address  Unreatistic  Initiai 
Cost  Estimates  of  Space  Systems,  GAO-07-96,  (Washington,  D.C.:  Nov.  17,  2006). 

^®GAO,  Defense  Acquisitions:  Assessments  ofSetected  Weapon  Programs, 

GAO-1 5-342SP,  (Washington,  D.C:  March  12,  2015). 
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customer  requirements  as  well  as  cost,  schedule,  and  reliability  targets. 
As  part  of  that  work,  we  also  found  that  a  knowledge  deficit  early  in  a 
program  can  cascade  through  design  and  production,  leaving  decision 
makers  with  less  knowledge  to  support  decisions  about  when  and  how 
best  to  move  into  subsequent  acquisition  phases  that  commit  more 
budgetary  resources.  Additionally,  we  found  that  demonstrating  a  stable 
and  mature  design,  typically  via  a  CDR,  is  generally  considered  a 
prerequisite  by  leading  commercial  firms  and  successful  DOD  programs 
for  moving  forward  with  a  program  because  it  assesses  final  product 
design  and  provides  assurance  that  product  specifications  have  been 
captured  in  detailed  design  documentation. 

The  201 1  preliminary  strategy  for  MGUE  called  for  development  of 
separate  ground,  air,  and  sea  receiver  cards  within  a  traditional  DOD 
three-phase  acquisition  process  that  encompassed  separate  technology 
development,  system  development,  and  production  phases.  The 
program’s  2012  baseline  strategy  was  similar  and  rolled  development  of 
air  and  sea  cards  together  based  on  the  Navy’s  assessment  that  aviation 
card  development  could  support  the  maritime  environment.  In  2014,  the 
Air  Force  received  approval  from  USD  AT&L  to  revise  the  MGUE 
acquisition  strategy  again  by  bypassing  a  formal  system  development 
phase  and  combining  the  development  and  production  decision  points. 
USD  AT&L  approved  the  associated  Air  Force  acquisition  strategy 
document  in  April  2015,  but  just  two  months  later  an  official  from  AT&L 
indicated  that — based  on  a  June  2015  memorandum  from  the  Assistant 
Secretary  of  Defense  for  Acquisition — those  milestones  would  be  split 
apart  again.  According  to  the  AT&L  official,  it  was  determined  to  be 
impractical  to  combine  the  milestones  because  there  were  many  events 
that  needed  to  be  conducted  between  them.  The  official  indicated  that 
USD  AT&L  plans  to  oversee  the  program  until  the  development  decision 
point,  at  which  time  it  would  then  delegate  program  oversight  to  the  Air 
Force. 

The  last  MGUE  acquisition  strategy  of  record  eliminated  a  key 
assessment  of  MGUE  design  maturity,  the  CDR,  which  we  have  found  in 
our  prior  best  practices  work  is  typically  held  mid-way  through  a 
program’s  development  phase  and  is  essentially  an  assessment  of 
whether  a  product’s  design  meets  the  customer’s  requirements.''®  The  Air 
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Force  stated  that  a  CDR  was  unnecessary  because,  among  other  things, 
detailed  design  work  normally  approved  at  CDR  was  completed  for 
MGUE’s  PDR  in  September  2014  and  hardware  and  software  designs 
and  cost,  schedule,  and  performance  risks  typically  reviewed  at  CDR 
were  assessed  as  part  of  that  PDR.  It  also  anticipated  other  events  would 
cover  and  go  beyond  the  purpose  of  a  CDR,  including  a  compatibility 
certification  process  to  ensure  MGUE  receiver  cards  were  compatible 
with  the  GPS  satellite  signal;  security  validation  to  ensure  contractors’ 
designs  met  security  certification  criteria;  initial  and  final  verification 
reviews  to  assess  product  maturity  and  deficiencies;  and  utilizing  multiple 
contractors  with  multiple  designs  from  which  the  military  services  can 
choose  to  meet  their  requirements. 

However,  the  MGUE  program  carries  design-related  risks  that  would 
typically  be  revisited  within  the  CDR  context.  An  April  2015  assessment  of 
the  MGUE  PDR  by  the  Office  of  the  Undersecretary  of  Defense  for 
System  Engineering  noted,  for  example,  that  some  security  and 
information  assurance  design  details  were  not  addressed  as  part  of  PDR 
and  had  been  deferred  to  the  security  verification  review  scheduled  to 
finish  in  late  summer/early  fall  2015.  The  Undersecretary’s  office  also 
pointed  out  that  the  refinement  of  security  countermeasures  may  result  in 
later  design  changes.  In  addition,  the  office  stated  that  MGUE  PDR 
interface  designs  with  the  military  services’  lead  platforms  may  not  be 
rigorous  enough  to  account  for  implementing  those  designs  across 
various  operating  environments. 

Moreover,  it  is  unclear  whether  MGUE  designs  meet  the  military  services’ 
requirements.  According  to  Army  program  officials,  the  Army  has 
identified  a  set  of  performance  gaps  for  ground  and  aviation  receiver 
cards  between  the  MGUE  program’s  technical  requirements  document 
and  Army  operational  requirements,  including  power  and  thermal 
incompatibility  issues  between  the  ground  receiver  card  and  the  platform 
the  cards  are  to  support.  The  MGUE  program  said  that  it  is  unaware  of 
any  data  showing  the  ground  cards  do  not  meet  power  and  thermal 
requirements;  analysis  shows  the  cards  meet  service  requirements  for 
power  as  captured  in  MGUE’s  capability  development  document;  the 
power  and  thermal  issues  may  be  caused  by  the  host  platforms  rather 
than  the  ground  cards;  and  as  such,  the  lead  platform  program  offices 
would  need  to  make  modification  to  their  power  supplies  or  platforms  to 
resolve  the  issues.  Army  officials,  in  contrast,  said  that  the  MGUE 
capability  development  document  does  not  contain  specific  power 
consumption  limits  and  that  the  Air  Force’s  GPS  Directorate  had  recently 
changed  power  limitations  by  unilaterally  editing  the  MGUE  technical 
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requirements  document,  against  significant  objection  by  the  Army.  A 
DOT&E  official  explained  to  us  that  the  Air  Force  believes  it  has 
addressed  this  issue  by  clarifying  the  specification  to  be  an  average 
maximum  power  requirement  for  MGUE  ground  cards  but  also 
emphasized  that  this  clarification  relieves  the  MGUE  contractors  of 
meeting  more  stringent  instantaneous  power  limits.  As  a  result,  the  official 
said,  there  are  potential  adverse  effects  to  host  platform  interface  designs 
which  may  then  require  additional  development  and  integration  by  the 
military  services  in  order  to  adapt  their  platforms  to  MGUE. 


Military  Services  Are 
Unlikely  to  Have  Complete 
Knowledge  of  MGUE 
Performance  by  Fiscal 
Year  2018 


Based  on  the  MGUE  program’s  April  2015  acquisition  strategy  document, 
as  shown  in  Figure  9  below,  formal  integration  and  operational  testing  for 
the  MGUE  aviation  receiver  card  on  the  Air  Force’s  B-2  aircraft  abuts 
fiscal  year  2018;  that  same  effort  for  the  Army’s  DAGR  DS/Stryker 
extends  into  fiscal  year  201 8;  and  it  carries  into  fiscal  year  2019  on  the 
GPNTS/Navy’s  DDG-51  Arleigh  Burke-class  destroyer  and  Marine  Corps’ 
JLTV.  In  accordance  with  DOD  guidance,  these  operational  tests  mark  an 
important  point  in  the  assessment  of  MGUE  design  and  performance — 
the  first  time  production  representative  test  articles  are  planned  to  be 
incorporated  on  the  lead  platforms  and  tested  in  a  realistic  environment. 
Among  other  things,  per  DOD  guidance  on  test  and  evaluation,  the 
operational  tests  are  expected  to  serve  as  field  tests  that  assess  the 
receiver  cards’  ability  to  satisfy  the  military  services’  requirements;  current 


The  military  services  are  unlikely  to  have  the  knowledge  to  begin 
informed  MGUE  procurement  at  the  start  of  fiscal  year  201 8  because 
operational  testing  that  will  provide  knowledge  about  MGUE’s  operational 
effectiveness  and  suitability  for  ground,  air,  and  sea  environments  will  not 
yet  have  been  completed. Prior  to  operational  testing,  the  Air  Force  is 
conducting  activities  and  demonstrations  that  it  believes  reduce  program 
risk  by  providing  knowledge  about  MGUE  performance.  Those  efforts, 
however,  have  limitations  which  call  into  question  their  value  in  assessing 
MGUE’s  ability  to  meet  the  military  services’  needs. 


20 

Operational  effectiveness  is  the  overall  degree  of  mission  accomplishment  of  a  system 
when  used  by  representative  personnel  (e.g.,  warfighters)  in  the  environment  planned  or 
expected  for  operational  employment  of  the  system  considering  organization,  training, 
doctrine,  tactics,  survivability  or  operational  security,  vulnerability,  and  threat.  Operational 
suitability  defines  the  degree  in  which  a  system  can  be  satisfactorily  placed  in  field  use, 
with  consideration  given  to  reliability,  transportability,  interoperability,  and  safety,  among 
other  attributes.  Defense  Acquisition  Guidebook,  oh. 9,  pars.  9. 3.2.1  and  9. 3. 2. 2. 
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capabilities,  including  operational  benefits  or  burdens;  and  the  need  for 
further  development  of  the  cards  to  correct  performance  deficiencies. 


Figure  9:  Time  Lines  for  MGUE  Formai  integration  and  Testing  on  Miiitary  Service 
Lead  Piatforms 
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The  Air  Force  believes  that  completion  of  operational  testing  is  not 
required  for  the  military  services  to  begin  informed  procurement  planning 
because  data  from  prior  test  events  can  be  leveraged.  It  stated  that  the 
MGUE  program  has  participated  in  various  testing  and  integration  risk 
reduction  activities  prior  to  the  time  lines  shown  in  figure  9,  and  that  those 
efforts  can  count  towards  MGUE  integration.  Those  activities  include, 
according  to  the  Air  Force,  successful  demonstration  of  technologies  on 
the  Army’s  Raven  unmanned  aerial  vehicle;  demonstration  of  MGUE 
receiver  cards  in  operationally  relevant  environments  on  a  surrogate  C-12 
aircraft  flying  similarly  enough  to  the  B-2  lead  platform  to  serve  as  a 
surrogate;  and  lead  platform-based  risk  reduction  events  via  prototype 
GPS  units  that  will  be  incorporated  into  the  B-2.  The  Air  Force  said  that 
testing  on  surrogate  platforms  is  intended  as  early  risk  reduction  activity 
rather  than  emulation  of  lead  platform  environments.  Additionally,  it  stated 
that  it  plans  to  conduct  subsystem-level  integration  on  lead  platforms  prior 
to  system-level  integration,  and  that  these  efforts  also  would  provide  the 
military  services  with  further  information  to  asses  MGUE  performance. 
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Some  of  the  military  services,  however,  believe  that  the  integration 
activities  to  date  have  been  more  limited  than  indicated  by  the  Air  Force 
and  that  they  are  now  responsible  for  more  of  the  MGUE  development 
effort  than  indicated  earlier  in  DOD  policy.  Army  officials  stated,  for 
example,  that  the  MGUE  program  began  initial  contact  with  the  DAGR  D3 
program  office  in  fiscal  year  2014,  but  that  integration  with  the  DAGR 
D3/Stryker  per  se  has  not  yet  started  due  to  lack  of  MGUE  devices  for 
platform  integration.  Instead,  they  said,  “fit  check”  tests  were  conducted  in 
early  2015  and  showed  the  DAGR  D3  is  unable  to  provide  sufficient 
power  to  two  of  the  three  MGUE  contractors’  ground  receiver  cards.  The 
officials  also  said  that  the  Army  planned  to  perform  thermal  testing  with 
the  DAGR  D3/Stryker,  but  such  testing  has  not  yet  been  executed  due  to 
lack  of  MGUE  devices.  They  emphasized  that  they  have  begun  planning 
for  MGUE  procurement  but  cannot  initiate  actual  procurement  at  the  start 
of  fiscal  year  2018  for  several  reasons,  including  the  need  to  modify  as- 
built  MGUE  devices  to  close  performance  gaps,  resolve  power  and 
thermal  issues  by  either  modifying  one  MGUE  card  design  or  the 
integration  on  some  100  platforms;  and  developing  and  implementing  a 
means  for  MGUE  and  the  currently-fielded  Selective  Availability  Anti- 
Spoofing  Module  GPS  receiver  to  coexist  and  interact  in  the  field.  In 
addition,  both  Army  and  Navy  officials  stated  that,  based  on  their 
understanding  of  DOD’s  2006  policy  on  GPS  user  equipment  and 
procurement,  the  Air  Force  was  to  develop  MGUE  to  production-ready 
status.  They  noted  that,  in  order  to  bring  MGUE  to  true  production-ready 
status,  their  respective  military  services  will  need  to  do  further 
development  after  the  Air  Force  completes  its  MGUE  development  effort. 

Moreover,  the  demonstrations  and  risk  reduction  activities  that  the  Air 
Force  points  to  may  not  establish  MGUE  performance  to  the  extent  that 
the  Air  Force  believes.  For  example,  DOT&E — the  DOD  independent  test 
authority  charged  with  ensuring  a  program’s  operational  testing  confirms 
operational  effectiveness  and  suitability  in  combat  use — expressed 
serious  concerns  about  the  Air  Force’s  characterization  of  the  results  from 
one  of  the  past  demonstrations.  In  a  November  2014  memorandum  to 
USD  AT&L,  DOT&E  emphasized  that  the  Air  Force  had  overstated  MGUE 
development  maturity  and  that  the  demonstrations  to-date  had  achieved 
more  mixed  results  than  the  Air  Force  indicated.  DOT&E  noted,  for 
example,  that  the  testing  with  the  Army’s  Raven  did  not  use  final 
development  models;  had  employed  a  limited  subset  of  realistic  threats; 
successfully  flew  its  intended  flight  pattern  in  just  7  out  of  51  attempts; 
and  that  none  of  those  flights  was  conducted  in  the  presence  of  electronic 
jamming.  It  went  on  to  say  that  developmental  activity  with  the  Raven 
should  in  no  way  be  construed  as  having  the  level  of  rigor  of  operational 
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testing  or  even  formal  developmental  testing.  Furthermore,  Army  program 
officials  stated  that  the  Army’s  lead  platform  had  been  changed  from  the 
Raven  to  the  DAGR  D3/Stryker  not  only  because  the  Army  does  not 
intend  to  procure  additional  Ravens  but  also  because  the  Raven  does  not 
adequately  stress  the  capabilities  of  the  ground  receiver  card. 

In  addition,  DOT&E  concluded  in  its  2014  memorandum  that  MGUE 
cannot  be  considered  effective  until  it  is  successfully  integrated  on  host 
platforms.  According  to  a  DOT&E  official,  subsystem  integration  does  not 
guarantee  proper  integration  into  a  larger  system  and  may  not  necessarily 
demonstrate  integration  and  performance  in  the  intended  operational 
environment — particularly  in  light  of  the  fact  that  the  MGUE  test  strategy 
is  inherently  risky  in  that  only  4  lead  platforms  are  designated  to  represent 
the  operating  conditions  for  over  100  platforms  service-wide.  He  also 
emphasized — as  did  DOT&E’s  2014  memorandum  — that  DOD  has  found 
integration  historically  challenging. 

Our  prior  work  on  integration  and  testing  has  also  shown  that  integration 
is  a  common  risk  in  system  development,  and  that  commercial  firms 
reduce  risk  by  capturing  design  and  manufacturing  knowledge  early. 

Our  previous  work  pointed  out,  for  example,  that  systems  integration 
problems  can  occur  even  though  the  various  components  performed 
successfully  on  previous  systems.  We  have  found,  in  illustration,  that  the 
Air  Force’s  C-17  aircraft  program  intended  to  use  current,  available,  and 
proven  technology  to  minimize  development  costs  and  structure  a  low 
technical  risk  effort  but  that  the  integration  of  technologies  was  a  major 
engineering  and  management  task  that  eventually  contributed  to 
significant  cost  increases. In  addition,  we  have  also  determined  that  it  is 
during  integration  and  testing  that  problems  are  likely  to  be  found. 
Furthermore,  as  we  pointed  out  in  a  2000  study  of  best  acquisition 
practices  for  testing  among  commercial  firms,  test  weaknesses  invariably 


^Vor  example,  GAO-02-701;  GAO-1 5-60;  GAO,  Access  to  Space:  Issues  Associated  with 
DOD’s  Evolved  Expendable  Launch  Vehicle  Program,  GAO/NSIAD-97-130  (Washington, 
D.C.:  June  24,  1997);  Defense  Acquisitions:  Incentives  and  Pressures  That  Drive 
Problems  Affecting  Satellite  and  Related  Acquisitions,  GAO-05-570R  (Washington,  D.C.: 
June  23,2005). 

22gAO/NSIAD-97-130. 

^^GAO,  NASA:  Assessments  of  Selected  Large-Scale  Projects,  GAO-1 1-239SP, 
(Washington,  D.C.:  Mar.  3,  2011). 
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cause  negative  program  outcomes,  such  as  cost  increases,  schedule 
delays,  or  performance  shortfalls. Our  work  on  best  commercial 
acquisition  practices  also  shows  that  such  negative  outcomes  can  be 
avoided  by  accumulating  knowledge  prior  to  beginning  production.  In  a 
2002  study  of  best  commercial  practices,  for  example,  we  found  that 
leading  commercial  firms  reduce  program  risk  by  demonstrating  fully- 
integrated  prototypes  prior  to  making  production  decisions.  Such 
demonstrations,  we  determined,  help  the  firms  decide  when  to  make  the 
transition  from  product  development  to  production  and  to  ensure  that 
transition  is  smooth. 

The  military  services  can  postpone  procuring  MGUE  if  they  lack  sufficient 
knowledge  about  MGUE  design  and  performance  after  fiscal  year  2017. 
They  can,  as  provided  by  the  National  Defense  Authorization  Act  for 
Fiscal  Year  201 1 ,  request  a  waiver  of  such  procurement.  Specifically,  the 
Secretary  of  Defense  may  waive  this  limitation  upon  a  determination  that 
suitable  M-code  capable  user  equipment  is  either  not  available  or  DOD 
does  not  require  that  user  equipment  be  capable  of  receiving  M-code 
from  GPS.^®  An  official  from  USD  AT&L  stated  that  the  waiver  process 
has  not  yet  been  established. 


^■^GAO,  Best  Practices:  A  More  Constructive  Approach  is  Key  to  Better  Weapon  System 
Outcomes,  GAO/NSIAD-00-199  (Washington,  D.C.:  July  31, 2000). 

2^GAO-02-701. 

^^Specifically,  the  limitation  is  on  the  use  of  funds  for  purchasing  GPS  user  equipment 
after  fiscal  year  2017  unless  the  equipment  is  capable  of  receiving  the  M-code  from  GPS. 
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GPS  Constellation 
Has  Been  More 
Reliable  Than 
Expected,  but  M-code 
Deployment  Is  Likely 
More  Than  a  Decade 
Away 


In  2010,  we  examined  GPS  satellite  reliability  data  and  reported  that  a 
delay  of  2  or  more  years  to  the  launches  of  GPS  III  satellites — originally 
projected  to  begin  launching  in  April  2014 — would  likely  reduce  the 
constellation  size  below  the  minimum  requirement  of  24  operational  GPS 
satellites  by  fiscal  year  2018.^^  Since  that  report,  while  the  GPS  III  and 
OCX  programs  have  been  delayed,  the  GPS  constellation  has  proven  to 
be  more  reliable  than  previously  expected  because  the  Air  Force 
successfully  extended  the  life  of  existing  satellites,  primarily  by  modifying 
the  satellites’  battery  charging  procedures.  However,  the  Air  Force  now 
needs  OCX  block  1  to  be  operational  by  late  2019  to  launch  and 
incorporate  GPS  Ills  into  the  constellation.  Given  the  ongoing 
development  problems  with  OCX,  delivery  by  that  date — and 
consequently,  overall  constellation  reliability — is  at  some  risk.  The  Air 
Force  is  exploring  contingency  plans,  including  modifying  the  current 
ground  system  to  launch  and  operate  GPS  Ills.  However,  the  modified 
ground  system  would  operate  GPS  III  satellites  as  current  legacy  GPS 
satellites,  thereby  not  utilizing  all  the  capabilities  on  the  new  satellites. 
Moreover,  this  approach  would  increase  the  risk  that  several  Ills  may  be 
launched  before  they  can  be  fully  tested  with  OCX.  Delays  to  OCX 
delivery  also  mean  that  M-code  functionality — which  requires  OCX  block 
1  to  be  fully  enabled — is  not  scheduled  to  be  deployed  until  2019  at  the 
earliest,  and  likely  for  another  decade  or  more  until  the  military  services 
can  widely  deploy  modernized  MGUE  receivers. 


The  Air  Force  Has  Found 
Ways  to  Extend  the  Life  of 
the  GPS  Constellation 


The  performance  standards  for  (1)  the  standard  positioning  service 
provided  to  civil  and  commercial  GPS  users,  and  (2)  the  precise 
positioning  service  provided  to  military  GPS  users,  commit  the  U.S. 
government  to  at  least  a  95  percent  probability  of  maintaining  a 
constellation  of  24  operational  GPS  satellites.  In  September  2010,  we 
reported  that  delays  to  the  launch  of  the  GPS  III  satellites — originally 
projected  to  begin  in  April  2014 — could  affect  the  long-term  probability  of 
maintaining  the  minimum  constellation  size.^®  Based  on  satellite  reliability 
data  provided  by  the  Air  Force,  we  had  predicted  that,  for  example,  a  2- 
year  delay  in  the  production  and  launch  of  the  first  and  all  subsequent 
GPS  III  satellites  would  reduce  the  guaranteed  size  of  the  constellation 
(at  the  95  percent  confidence  level)  to  about  18  satellites  by  around  fiscal 
year  201 8. 


2^GA0-1  0-636. 
2®GA0-1  0-636. 
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Since  our  201 0  report,  the  launch  availability  date  for  the  first  GPS  III 
satellite  has,  in  fact,  slipped  by  about  3  years,  to  May  2017,  mainly  due  to 
development  difficulties  with  its  navigation  payload.  Moreover,  as 
discussed  earlier,  the  delivery  of  OCX  block  1 — required  to  operate  the 
Ills — has  been  delayed  by  about  4  years,  to  mid-2019.^®  However,  the  3- 
year  delay  in  GPS  Ill’s  delivery  has  not  resulted  in  a  predicted  significant 
reduction  in  the  size  of  the  GPS  constellation  largely  because  the 
expected  life  of  the  existing  generations  of  GPS  satellites  has  risen 
dramatically.  Figure  10  below  displays  the  probability  of  maintaining  a  24- 
satellite  GPS  constellation  as  a  function  of  time  based  on  March  2014 
satellite  reliability  data  and  launch  schedules — the  latest  date  for  which  a 
complete  set  of  approved  parameters  is  available.  The  launch  schedule 
has  been  adjusted  to  reflect  the  successful  launches  of  four  GPS  IIF 
satellites  through  June  2015. 


Figure  10:  Probability  of  Maintaining  a  Constellation  of  at  Least  24  GPS  Satellites  Based  on  Reliability  and  Launch  Schedule 
as  of  March  2014 
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-  Predicted  probability  of  maintaining  24-satellite  constellation. 

-  Committed  probability  of  maintaining  24-satellite  constellation. 

Source:  GAO  analysis  of  DOD  data.  |  GAO-15-657 


^®While  GPS  III  satellites  can  be  launched  and  initially  tested  using  OCX  block  0,  OCX 
block  1  capabilities  are  required  to  operate  the  GPS  III  satellites. 


Page  36 


GAO-15-657  GPS 


Expected  Life  of  Current  GPS 
Satellites  Has  Risen  Sharply 


The  expected  lifetimes  of  the  GPS  MR  and  IIR-M  satellites — the  types  of 
satellites  that  currently  comprise  the  majority  of  the  current 
constellation — have  risen  dramatically  since  our  2010  report,  from  14.8 
and  10.5  years,  respectively,  to  20.4  and  17.5  years,  respectively. This 
increase  in  expected  lifetime  is  due  mainly  to  the  Air  Force’s 
implementation  of  a  modified  charging  procedure  that  will  prolong  the  life 
of  the  satellites’  batteries.  Spacecraft  batteries — like  all  other 
rechargeable  batteries — are  subject  to  cycle  life,  that  is,  they  can  only  be 
charged  and  discharged  so  many  times  before  they  will  not  charge 
efficiently  anymore.  Because  satellite  batteries  were  predicted  to  be  one 
of  the  primary  life-limiting  components  of  these  satellites,  the  Air  Force 
developed  and  tested  a  procedure  in  2012  to  reduce  the  battery  charge 
rates  during  certain  times  of  the  year.  The  Air  Force  estimated  that  this 
new  procedure  would  increase  the  expected  lifetimes  of  the  MR  and  MR-M 
satellites  by  1  to  2  years.  Based  on  recent  telemetry  data,  the  new 
procedure  has  proved  successful. In  2014,  the  Air  Force  announced 
that  this  modified  battery  charging  procedure  has  added,  in  aggregate,  27 
additional  years  of  operational  life  to  the  GPS  MR  and  MR-M  satellites. 
Moreover,  according  to  Air  Force  representatives,  additional  information 
suggests  that  this  estimate  is  probably  conservative,  and  that  the  next 
formal  review  of  satellite  reliability  data  will  likely  reveal  an  additional 
increase  in  the  expected  life  of  the  MR  satellites.  Because  these  satellites 
form  the  mainstay  of  the  current  GPS  constellation,  and  because  most  of 
them  are  still  expected  to  be  operational  in  2017,  when  the  first  GPS  Ml 
satellite  is  planned  to  be  available  for  launch,  an  increase  in  the  expected 
life  of  these  older  satellites  significantly  improves  the  Air  Force’s  ability  to 
meet  the  24-satellite  performance  standard  despite  the  3-year  slip  in  GPS 
Ml  and  4-year  slip  in  OCX. 

The  expected  lifetimes  of  the  GPS  MA  satellites — the  oldest  version  of 
GPS  satellites  on  orbit — have  also  risen  since  our  2010  report,  from  16.5 
years  to  19.7  years,  due  to  positive  trends  in  on  orbit  performance. 
However,  the  effect  of  this  increase  on  the  Air  Force’s  ability  to  meet  the 
24-satellite  performance  standards  is  less  significant  than  the  effect  of  the 


OA 

The  IIR-M  satellites  are  a  variant  of  the  MR  series  and  are  the  first  satellites  capable  of 
broadcasting  the  new  M-code  signal. 

^^Telemetry  data  consists  of  health  and  status  reports  which  are  transmitted  from  the 
satellite  to  the  ground. 


Page  37 


GAO-15-657  GPS 


IIR/IIR-M  life  expectancy  increase. The  current  constellation  availability 
analysis  assumes  each  of  these  satellites  is  only  available  to  replace 
other  satellites  in  the  unlikely  event  of  a  large  number  of  satellite  failures 
until  OCX  is  operational.  Because  OCX  is  not  expected  to  be  capable  of 
operating  the  IIA  satellites,  the  analysis  assumes  that  any  remaining  IIA 
satellites  would  be  turned  off  by  July  2019,  the  date  that  OCX  is 
scheduled  to  be  required  to  be  ready  to  transition  to  operations. 


Additional  OCX  Delays 
Could  Affect  Constellation 
Availability 


Despite  the  delays  to  GPS  III  and  OCX,  the  GPS  constellation  is 
expected  to  meet  the  constellation  size  requirement  if  OCX  block  1  is 
available  by  July  2019  as  needed.  However,  further  OCX  delays  could 
affect  overall  constellation  availability,  bringing  it  below  the  95-percent 
performance  standard  for  maintaining  a  24-satellite  constellation  for  a 
year  or  more.  For  example,  if  OCX  block  1  is  not  ready  to  transition  to 
operations  until  November  2020 — as  the  most  recent  (October  2014) 
independent  review  of  OCX  estimated — the  Air  Force  could  experience  a 
15-month  period  during  which  it  would  not  meet  its  commitment  to 
maintaining  a  24-satellite  constellation  with  a  95  percent  confidence  level. 
While  GPS  III  satellites  could  still  be  launched  beginning  in  2017,  they 
would  not  be  added  to  the  constellation  until  OCX  comes  on  line  or  until  a 
contingency  operational  control  capability  able  to  command  GPS  III 
satellites  is  available.  Figure  11  depicts  the  predicted  constellation  size 
based  on  OCX  block  1  being  potentially  delayed  until  November  2020. 


oo 

Because  the  maximum  size  of  the  current  constellation  is  limited  to  31  operational 
satellites,  most  of  these  remaining  IIA  satellites  have  been  placed  in  a  residual  status  as 
the  newer  series  of  GPS  IIP  satellites  has  begun  to  be  launched  and  added  to  the 
operational  constellation.  While  several  IIA  satellites  are  likely  to  still  be  functioning  in 
2017,  when  the  first  GPS  III  is  available  for  launch,  and  may  even  be  functioning  in  mid- 
2019,  when  OCX  block  1  is  scheduled  to  be  delivered,  most  or  all  of  these  satellites  are 
expected  to  be  placed  in  a  residual  status  by  that  time,  and  will  not  be  reactivated  unless 
the  constellation  experiences  a  much  higher  than  expected  rate  of  MR  or  IIP  failures. 
Moreover,  according  to  Air  Porce  representatives,  while  these  satellites  are  still  viable  and 
could  be  used  under  “worst  case”  circumstances,  each  of  them  is  well  beyond  its  design 
life,  and  some  have  performance  problems  that  would  make  their  use  in  the  operational 
constellation  challenging. 
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Figure  11:  Probability  of  Maintaining  a  Constellation  of  at  Least  24  GPS  Satellites  Based  on  Reliability  and  Launch  Schedule 
as  of  March  2014,  OCX  Delayed  Until  November  2020 

Probability 


-  Predicted  probability  of  maintaining  24-satellite  constellation. 

-  Committed  probability  of  maintaining  24-satellite  constellation. 

Source:  GAO  analysis  of  DOD  data  |  GAO-15-657 


The  Air  Force  Is  Taking 
Actions  to  Sustain  the 
GPS  Constellation  Given 
Delays  to  OCX,  but  These 
Efforts  Will  Not  Fully 
Exploit  GPS  Capabilities 


GPS  capabilities  are  unbalanced,  with  satellite  capabilities  outstripping 
those  supported  by  the  ground  and  user  equipment  segments.  In 
particular,  the  Air  Force  has  launched  two  generations  of  satellites — IIR-M 
and  IIF — capabilities  of  which  are  still  not  fully  utilized  because  they  have 
been  transmitting  three  signals,  including  M-code,  that  are  not  supported 
by  the  current  GPS  ground  control  system,  OCS. 

GPS  IIFs  are  nearing  the  end  of  their  launch,  with  the  last  three 
scheduled  to  be  launched  between  July  2015  and  February  2016.  GPS 
Ills  are  needed  to  continue  to  sustain  the  constellation.  Since  OCX  block 
1  is  needed  to  operate  the  GPS  III  satellites  as  part  of  the  constellation, 
the  Air  Force  is  preparing  contingency  plans  in  the  event  that  OCX  block 
1  is  delayed  beyond  August  2019,  the  need  date  for  the  first  GPS  III 
satellite.  The  Air  Force  has  pinpointed  one  option  as  the  most  viable — 
modifying  the  current  OCS  system  to  operate  the  GPS  III  satellites  but  at 
the  older,  IIF  satellite  level  of  functionality;  this  option  would  deliver  M- 
code  support  without  all  modernized  GPS  III  functionality,  specifically 
support  for  the  L1C  signal.  Under  this  plan,  the  Air  Force  would  use  OCX 
block  0  to  launch  and  initially  test  the  GPS  III  satellites,  and  use  the 
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modified  OCS  to  control  the  satellites’  navigation  signals  as  part  of  the 
overall  constellation.  As  figure  12  shows,  successfully  implementing  this 
contingency  plan  should  enable  the  Air  Force  to  meet  its  commitment  to 
maintaining  a  24-satellite  constellation  with  a  95  percent  confidence  level, 
even  if  the  OCX  block  1  delivery  date  were  to  slip  to  November  2020  (or 
later),  but  the  plan  would  require  funding  and  other  resources. 


Figure  12:  Probability  of  Maintaining  a  Constellation  of  at  Least  24  GPS  Satellites  Assuming  Contingency  Plans  to  Launch  Ills 
if  OCX  Is  Delayed  until  November  2020 

Probability 


0.9 


0.8 


0.7 


-  Predicted  probability  of  maintaining  24-satellite  constellation. 

-  Committed  probability  of  maintaining  24-satellite  constellation. 

Source:  GAO  analysis  of  DOD  data.  |  GAO-15-657 


The  Air  Force  estimated  the  cost  of  this  option  at  $105  million  as  of 
February  2015;  however,  pursuing  this  option  means  that  the  Air  Force 
may  launch  up  to  5  GPS  III  satellites  without  fully  testing  them  with  OCX 
block  1 ,  increasing  the  risk  that  issues  may  be  found  during  testing 
without  the  ability  to  fix  the  satellites  already  launched. 
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Initial  M-code  Capability 
Will  Not  Be  Available  until 
OCX  Delivery  in  Mid-2019, 
and  Full  M-code 
Deployment  Is  Likely  at 
Least  Another  Decade 
Away 


M-code  initial  operational  capability  is  defined  as  having  18  M-code 
capable  satellites  on  orbit,  the  control  segment  able  to  command/upload 
M-code  capabilities  to  the  space  segment,  and  MGUE  receivers  fielded 
across  the  military  services  to  utilize  M-code  capabilities  operationally. 

Full  operational  capability  occurs  when  24  M-code  satellites  are  on  orbit, 
and  a  larger  portion  of  MGUE  is  fielded. 

The  GPS  constellation  will  likely  include  18  M-code  capable  satellites  by 
September  2015,  but  M-code  capabilities  will  not  be  available  to  users — 
assuming  MGUE  receivers  can  be  fielded — until  OCX  delivery  in  mid- 
2019.  The  launching  of  M-code  capable  satellites  has  been  stretched 
over  a  much  longer  period  than  originally  envisioned  because  of  delays  to 
the  IIF,  III,  OCX,  and  MGUE  programs.  However,  because  the  oldest  of 
the  M-code  capable  satellites  are  lasting  longer  than  originally  predicted, 
the  effect  of  this  stretched  deployment  has  been  mitigated.  In  particular 
the  oldest  M-code  capable  satellite — the  first  IIR-M  satellite,  launched  in 
September  2005 — has  a  better  than  a  75  probability  of  still  being 
operational  in  October  2020,  and  better  than  a  50  percent  probability  of 
still  being  operational  in  October  2023.  Nevertheless,  DOD  cannot  take 
full  advantage  of  M-code  capability  until  MGUE  receivers  are  deployed  in 
sufficient  numbers  across  the  services.  As  noted  earlier,  the  earliest 
MGUE  operational  test  will  not  be  completed  until  the  end  of  fiscal  year 
201 7  and  the  latest  at  the  end  of  fiscal  year  201 9.  Accordingly,  some  of 
the  military  services  will  not  be  able  to  make  informed  fielding  decisions 
until  fiscal  year  2020  at  the  earliest,  assuming  operational  testing  goes  as 
planned. 


Furthermore,  as  discussed  earlier,  the  Air  Force  plans  to  deliver 
production  representative  test  articles,  which  the  military  services  will  then 
acquire,  integrate,  and  operationally  test  only  on  selected  military 
platforms.  As  GAO  reported  in  2009,  each  of  the  military  services  would 
still  need  to  add  the  new  user  equipment  to  other  platforms,  which  could 
take  an  additional  10  or  more  years  based  on  the  need  to  perform  such 
activities  as  coordinating  installation  with  the  platforms’  maintenance  and 
upgrade  schedules. 


^^GAO-09-325. 
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Conclusions 


GPS  is  a  global  utility  that  is  integral  to  U.S.  national  security  and  civilian 
use.  As  a  result,  any  decision  about  GPS  has  far-reaching  consequences. 
A  combination  of  many  factors,  including  technical  challenges,  poor 
contractor  execution  and  program  management,  and  ineffective 
acquisition  oversight  have  all  put  GPS  modernization  at  significant  risk. 

OCX  is  the  key  to  enabling  the  full  GPS  capability  that  both  the  military 
and  civilian  communities  depend  on,  from  command  and  control  of  the 
satellite  constellation  including  the  new  GPS  III  satellites,  to  allowing 
military  receivers  to  take  advantage  of  M-code  signals  for  more  robust 
warfighting  capability,  to  enabling  advanced  civil  GPS  signals.  However, 
by  any  measure,  OCX  development  has  been  mired  in  development 
difficulties  resulting  in  steady  cost  growth  and  schedule  delays.  Moreover, 
despite  a  7  month  pause  ending  in  mid-2014,  OCX  has  yet  to  turn  the 
corner  on  resolving  the  problems  that  have  affected  the  program  since 
development  began  in  2010.  Independent  observers  such  as  OSD-led 
teams,  DCMA,  and  DOT&E  have  raised  red  flags  about  the  effectiveness 
of  the  Air  Force’s  oversight  and  Raytheon’s  ability  to  deliver  promised 
outcomes.  The  Air  Force  has  compounded  matters  by  consistently 
presenting  overly  optimistic  assessments  about  OCX  progress — 
demonstrating  a  pattern  of  marginalizing  warnings  about  OCX  delays 
presented  by  independent  assessors,  likely  because  their  assessments 
are  considered  advisory  in  nature.  Driving  towards  unrealistic  timeframes 
is  compounding  the  program’s  inability  to  meet  stated  cost  and  schedule 
goals.  Five  years  into  what  was  originally  estimated  to  be  a  5-year  effort, 
OCX  is  still  roughly  5  years  away  from  completion.  Without 
comprehensively  identifying  systemic  causes  for  OCX  problems,  DOD 
cannot  have  high-confidence  cost  and  schedule  estimates  for  OCX.  The 
Air  Force  and  Raytheon  have  noted  that  0TB  corrective  actions  will  begin 
paying  off  in  early  2016  as  block  1  development  resumes  in  earnest.  But 
given  the  lack  of  success  in  prior  attempts  to  turn  the  program  around,  the 
Air  Force  could  benefit  from  external  expertise  and  guidance  on  what  is 
necessary  to  address  systemic  issues. 

Since  the  enactment  of  the  statute  directing  DOD  to  generally  procure 
only  M-code  capable  user  equipment  after  fiscal  year  2017,  the  Air  Force 
has  struggled  to  deliver  a  MGUE  acquisition  strategy  that  would  allow  the 
military  services  to  comply  with  that  direction.  As  it  stands  now,  the  Air 
Force  essentially  truncates  its  MGUE  development  work  and  hands  the 
result  of  its  efforts  off  to  the  military  services  to  continue  development  on 
their  respective  platforms.  At  the  point  the  Air  Force  provides  funds  to  the 
military  services  for  them  to  acquire  and  test  MGUE,  it  also  transfers  the 
onus  of  the  development  work  to  the  military  services’  shoulders  because 
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MGUE  development  cannot  be  considered  complete  until  the  cards  work 
in  the  lead  platforms.  Unfortunately,  the  military  services  will  absorb  an 
added  development  burden  because  the  Air  Force  plans  to  transfer  its 
work  without  conducting  design  and  performance  assessments  that  could 
help  the  services  decide  the  extent  to  which  the  cards  are  ready  to 
integrate  and  test  and  how  much  additional  work  they  will  need  to  do 
before  beginning  to  procure  the  cards. 

Fortunately,  many  GPS  satellites  now  on  orbit  have  served  the  nation 
particularly  well  by  working  considerably  longer  than  expected.  Even  so,  it 
cannot  be  presumed  that  they  will  continue  to  do  so.  Therefore,  it  is 
critical  that  the  modernized  GPS  III,  OCX,  and  MGUE  development 
efforts  succeed  sooner  rather  than  later.  Of  those  three  programs,  OCX  is 
now  the  pacing  item  for  modernization  due  to  its  many  past  delays  and 
probable  future  delays.  Until  the  OCX  program  trajectory  is  corrected, 
those  delays  are  likely  to  pose  significant  risks  to  sustaining  the  GPS 
constellation,  and  consequently,  delivering  GPS  capability  to  the  military 
community. 


Recommendations 
Executive  Action 


•  Convene  an  independent  task  force  comprising  experts  from  other 
military  services  and  defense  agencies  with  substantial  knowledge 
and  expertise  to  provide  an  assessment  to  USD  AT&L  of  the  OCX 
program  and  concrete  guidance  for  addressing  the  OCX  program’s 
underlying  problems,  particularly  including: 


To  better  position  DOD  as  it  continues  pursuing  GPS  modernization,  to 
have  the  information  necessary  to  make  decisions  on  how  best  to 
improve  that  modernization,  and  to  mitigate  risks  to  sustaining  the  GPS 
constellation,  we  recommend  that  the  Secretary  of  Defense  take  the 
following  five  actions: 


•  A  detailed  engineering  assessment  of  OCX  defects  to  determine 
the  systemic  root  causes  of  the  defects; 

•  Whether  the  contractor’s  software  development  procedures  and 
practices  match  the  levels  described  in  the  OCX  systems 
engineering  and  software  development  plans;  and 

•  Whether  the  contractor  is  capable  of  executing  the  program  as 
currently  resourced  and  structured. 

•  Develop  high  confidence  OCX  cost  and  schedule  estimates  based  on 
actual  track  record  for  productivity  and  learning  curves. 
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•  Direct  the  Air  Force  to  retain  experts  from  the  independent  task  force 
as  a  management  advisory  team  to  assist  the  OCX  program  office  in 
conducting  regular  systemic  analysis  of  defects  and  to  help  ensure 
OCX  corrective  measures  are  implemented  successfully  and 
sustained. 

•  Put  in  place  a  mechanism  for  ensuring  that  the  knowledge  gained 
from  the  OCX  assessment  is  used  to  determine  whether  further 
programmatic  changes  are  needed  to  strengthen  oversight. 

•  To  allow  the  military  services  to  fully  assess  the  maturity  of  the  MODE 
design  before  committing  test  and  procurement  resources, 
incorporate  a  CDR  in  the  Air  Force’s  MODE  development  effort. 


Agency  Comments 
and  Our  Evaluation 


We  provided  a  draft  of  this  report  to  the  Department  of  Defense  for  review 
and  comment.  In  its  written  comments,  reproduced  in  appendix  II,  DOD 
concurred  with  the  four  OCX-related  recommendations  calling  for  a  more 
robust  independent  review  of  the  program,  stating  that  the  intent  of  those 
recommendations  has  been  met  by  OSD-chartered  independent  reviews 
conducted  to-date,  which  included  both  Air  Force  and  OSD  staff,  and 
other  DOD  activities  governing  OCX  oversight,  such  as  the  use  of 
independent  cost  estimates.  DOD  partially  concurred  with  the 
recommendation  calling  for  incorporating  a  CDR  in  the  MODE  program’s 
development  effort,  noting  that  adding  a  CDR  would  delay  the  program 
amidst  its  efforts  to  compress  the  acquisition  process  to  deliver  MODE 
capability  by  fiscal  year  2018.  DOD  also  provided  technical  comments, 
which  we  have  incorporated  as  appropriate. 

While  DOD  concurred  with  our  recommendations  to  put  OCX  on  a  better 
path  forward,  it  responded  that  it  had  essentially  been  following  the  intent 
of  our  recommendations  all  along,  requiring  no  further  action  from  the 
department.  These  comments  provide  little  confidence  that  the 
department  intends  to  fully  implement  our  recommendations  to  fix  the 
development  problems  that  have  beset  the  program  since  its  inception. 

As  we  noted  in  our  report,  the  independent  reviews  conducted  to-date 
have  been  nonbinding  and  advisory  in  nature.  Even  as  the  OSD-led 
reviews  have  warned  of  significant  delays  and  inadequate  insight  into 
systemic  root  causes  of  development  problems,  the  Air  Force  has 
repeatedly  developed  unrealistic  timeframes,  ostensibly  with  OSD’s 
knowledge.  If  business  continues  as  usual  without  swift  and  thoughtful 
action,  OCX  will  likely  continue  on  its  path  of  demonstrating  poor  cost  and 
schedule  outcomes.  We  continue  to  stand  by  our  recommendations 
calling  for  a  fresh  review — this  time  an  in-depth  and  comprehensive 


Page  44 


GAO-15-657  GPS 


critical  review  of  the  program — to  identify  the  true  root  causes  of  OCX 
development  difficulties  and  to  ensure  the  Air  Force  implements  the 
corrective  actions. 

Regarding  DOD’s  response  to  our  recommendation  that  DOD  hold  a  CDR 
for  MODE,  we  note  that  DOD’s  rationale  for  skipping  this  best  practice 
step  is  based  on  a  desire  to  accelerate  fielding  of  the  units.  However,  our 
past  work  has  consistently  shown  that  taking  shortcuts  and  skipping 
important  knowledge  points  in  the  acquisition  process  generally  results  in 
an  inability  to  deliver  promised  cost  and  schedule  outcomes.  To  minimize 
any  potential  disruption  to  development  efforts,  a  CDR  could  be  held  after 
production-representative  test  articles  are  delivered.  The  results  of  a  CDR 
will  show  whether  the  MGUE  design  is  stable.  Rushing  into  lead  platform 
testing  without  an  approved  and  stable  design  means  that  DOD  will  have 
to  concurrently  test  and  correct  the  receivers’  design  at  a  point  where 
resolving  issues  are  typically  more  expensive  than  earlier  in 
development — not  in  accordance  with  best  practices  that  are  important  to 
achieving  cost  and  schedule  outcomes  with  a  high  degree  of  confidence. 


We  are  sending  copies  of  this  report  to  the  appropriate  congressional 
committees,  the  Secretary  of  the  Defense,  the  Secretary  of  the  Air  Force, 
and  other  interested  parties.  In  addition,  the  report  is  available  at  no 
charge  on  the  GAO  website  at  http://www.gao.gov. 

If  you  or  your  staff  have  any  questions  concerning  this  report,  please 
contact  me  at  (202)  512-4841  or  by  email  at  chaplainc@gao.gov.  Contact 
points  for  our  Offices  of  Congressional  Relations  and  Public  Affairs  may 
be  found  on  the  last  page  of  this  report.  Key  contributors  to  this  report  are 
listed  in  appendix  III. 


Cristina  T.  Chaplain 

Director,  Acquisition  and  Sourcing  Management 
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List  of  Committees 


The  Honorable  John  McCain 
Chairman 

The  Honorable  Jack  Reed 
Ranking  Member 
Committee  on  Armed  Services 
United  States  Senate 

The  Honorable  Thad  Cochran 
Chairman 

The  Honorable  Richard  J.  Durbin 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
United  States  Senate 

The  Honorable  Mac  Thornberry 
Chairman 

The  Honorable  Adam  Smith 
Ranking  Member 
Committee  on  Armed  Services 
House  of  Representatives 

The  Honorable  Rodney  Frelinghuysen 
Chairman 

The  Honorable  Pete  Visclosky 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
House  of  Representatives 
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Appendix  I:  Objectives,  Scope,  and 
Methodology 


In  May  2014  House  Report  No.  113-446,  accompanying  H.R.  4435,  the 
Howard  P.  “Buck”  McKeon  National  Defense  Authorization  Act  for  Fiscal 
Year  2015,  noted  the  requirements  stated  in  section  913  of  the  Ike 
Skelton  National  Defense  Authorization  Act  for  Fiscal  Year  201 1  (Public 
Law  No.  1 11-383)  requiring  the  Department  of  Defense  (DOD)  to 
purchase  M-code  capable  user  equipment  during  the  fiscal  years  after 
fiscal  year  2017  and  included  a  provision  for  us  to  report  on  DOD’s 
progress  in  deploying  M-code  capability.  The  House  report  also  provided 
that  our  report  assess  current  and  planned  investments;  whether  key 
milestones  are  being  met;  the  projected  ability  to  the  meet  the 
requirements  in  section  913  of  Public  Law  No.  1 11-383;  and  an 
identification  of  the  challenges  that  Global  Positioning  System  (GPS) 
faces  and  possible  recommendations  on  how  to  make  the  program  more 
successful  in  delivering  M-code  capabilities.  Additionally,  in  June  2014, 
Senate  Report  No.  113-176,  accompanying  the  Carl  Levin  National 
Defense  Authorization  Act  for  Fiscal  Year  2015,  included  a  provision  for 
us  to  review  the  cost,  scope,  and  schedule  of  the  GPS  III  operational 
control  system,  including  synchronization  with  the  launch  of  the  GPS  III 
constellation  with  recommendations  for  improvement. 

In  response,  for  this  report  our  objectives  were  to  assess  (1 )  the  extent  to 
which  DOD  is  meeting  cost,  schedule,  and  performance  requirements  for 
next  generation  operational  control  system  (OCX),  (2)  the  progress  DOD 
has  made  in  delivering  M-code  capable  military  GPS  user  equipment 
(MGUE)  by  the  end  of  fiscal  year  201 7,  and  (3)  the  challenges  DOD  faces 
in  synchronizing  the  development  of  GPS  III  satellites,  OCX,  and  MGUE 
to  deploy  M-code. 

To  assess  the  extent  to  which  DOD  is  meeting  cost,  schedule,  and 
performance  requirements  for  OCX,  we  reviewed  program  and  contractor 
cost  and  schedule  documentation,  including  program  acquisition 
baselines,  earned  value  metrics,  and  test  plans.  Throughout  this  report, 
we  focused  on  the  costs  of  the  OCX  development  contract  instead  of  the 
full  program  acquisition  costs  because  the  latter  includes  prior, 
technology  development  expenditures  as  well  as  costs  of  management 
support  and  enterprise  integration  support  services.  We  analyzed  the 
progress  made  against  planned  program  milestones  and  reviewed 
technical  documentation  such  as  software  development  plans  to  gain 
insights  into  OCX  progress.  In  addition,  we  reviewed  briefings  and 
schedule  documentation  provided  by  program  and  contractor  officials  to 
determine  changes  in  OCX  cost,  schedule,  and  performance  overtime. 
These  documents  included  annual  GPS  enterprise  reviews,  OCX 
program  assessments,  and  program  status  briefings.  We  also  interviewed 
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officials  from  the  Air  Force  Space  and  Missile  Systems  Center  GPS  and 
OCX  program  offices,  OCX  prime  contractor  Raytheon,  and  Defense 
Contract  Management  Agency  (DCMA)  officials  charged  with  oversight  of 
the  OCX  contractor  efforts  to  identify  and  assess  cost  and  schedule 
issues  facing  the  program’s  development  efforts,  major  program  risks, 
and  technical  challenges.  Finally,  we  interviewed  officials  from  DOD’s 
Office  of  Cost  Assessment  and  Program  Evaluation  and  Director, 
Operational  Test  and  Evaluation  office  to  discuss  cost,  schedule,  and 
performance  challenges  for  OCX.  When  program  documents  identified 
program  events  by  fiscal  quarter  rather  than  by  month,  we  used  the  last 
month  of  the  given  quarter  as  the  date  of  the  event. 

To  determine  the  progress  DOD  has  made  in  delivering  M-code  capable 
MODE  by  the  end  of  fiscal  year  2017,  we  reviewed  and  analyzed  program 
plans  and  documentation  related  to  cost,  schedule,  acquisition  strategy, 
technology  development,  and  major  challenges  to  delivering  MODE 
Increment  1 .  We  then  compared  the  information  we  obtained  to  GAO’s 
criteria  for  best  practices  in  system  development.  To  assess  the 
program’s  progress  and  challenges,  we  held  discussions  with  and 
received  information  from  officials  at  the  Air  Force  Space  and  Missile 
Systems  Center  GPS  and  MGUE  program  offices;  Office  of  the 
Undersecretary  of  Defense  for  Acquisition,  Technology,  and  Logistics; 
DOD’s  Office  of  Cost  Assessment  and  Program  Evaluation;  and  DOD’s 
Director,  Operational  Test  and  Evaluation.  Additionally,  to  understand 
their  respective  development  efforts  and  challenges,  we  interviewed 
contractor  officials  from  the  three  MGUE  prime  contractors,  L-3  Interstate 
Electronics  Corporation,  Raytheon  Space  and  Airborne  Systems,  and 
Rockwell  Collins  Inc.  as  well  as  Defense  Contract  Management  Agency 
officials  overseeing  those  contractors.  To  identify  the  military  services’ 
concerns  about  MGUE  development,  schedule,  and  integration  and 
testing  with  their  respective  platforms,  we  interviewed  officials  from  the 
lead  platform  program  offices  for  the  Army’s  Defense  Advanced  GPS 
Receiver  Distributed  Device/Stryker,  Air  Force’s  B-2  aircraft,  Navy’s 
Arleigh  Burke  Class  DDG-51  ship,  and  Marine  Corps  Joint  Light  Tactical 
Vehicle.  As  with  OCX,  when  program  documents  identified  program 
events  by  fiscal  quarter  rather  than  by  month,  we  used  the  last  month  of 
the  given  quarter  as  the  date  of  the  event.  We  did  not  assess  MGUE 
Increment  2  because  it  has  not  yet  reached  its  technology  development 
milestone. 
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To  identify  the  challenges  faced  in  synchronizing  GPS  III  satellite,  OCX, 
and  MGUE  to  deploy  M-code  capability,  we  built  on  our  work  assessing 
the  plans,  schedules,  and  challenges  of  the  OCX  and  MGUE  programs  by 
reviewing  integrated  master  schedules  and  analyzing  approved  GPS 
constellation  reliability  parameters.  We  also  interviewed  cognizant 
officials  within  Air  Force  Space  Command  (AFSPC)  and  the  Aerospace 
Corporation,  which  supports  the  Air  Force’s  Space  and  Missile  Systems 
Center  (SMC),  to  evaluate  predicted  reliability  of  the  GPS  satellite 
constellation.  We  did  not  assess  cost,  schedule,  or  performance  for  GPS 
III  satellites. 

To  assess  the  status  of  the  GPS  constellation,  we  interviewed  officials 
from  the  Air  Force  SMC  GPS  program  office  and  AFSPC.  To  assess  the 
risks  that  a  delay  in  the  acquisition  and  fielding  of  GPS  III  satellites  could 
result  in  the  GPS  constellation  falling  below  the  24  satellites  required  by 
the  standard  positioning  service  and  precise  positioning  service 
performance  standards,  we  employed  a  methodology  very  similar  to  the 
one  we  had  used  to  assess  constellation  performance  in  2009  and  2010.'' 
We  obtained  information  from  the  Air  Force  predicting  the  reliability  for  57 
GPS  satellites — each  of  the  39  current  (on-orbit  as  of  March  2015)  and  18 
future  GPS  satellites — as  a  function  of  time.  Each  satellite’s  total  reliability 
curve  defines  the  probability  that  the  satellite  will  still  be  operational  at  a 
given  time  in  the  future.  It  is  generated  from  the  product  of  two  reliability 
curves — a  wear-out  reliability  curve  defined  by  the  cumulative  normal 
distribution,  and  a  random  reliability  curve  defined  by  the  cumulative 
Weibull  distribution.  For  each  of  the  57  satellites,  we  obtained  the  two 
parameters  defining  the  cumulative  normal  distribution,  and  the  two 
parameters  defining  the  cumulative  Weibull  distribution.  For  each  of  the 
18  unlaunched  satellites  we  included  in  our  model,  we  also  obtained  a 
parameter  defining  its  probability  of  successful  launch,  and  its  current 
scheduled  launch  date.  The  18  unlaunched  satellites  include  3  IIF 
satellites  and  15  III  satellites;  launch  of  the  final  III  satellite  we  included  in 
our  model  is  scheduled  for  March  2025.  Using  this  information,  we 
generated  overall  reliability  curves  for  each  of  the  57  GPS  satellites.  We 
discussed  with  Air  Force  and  Aerospace  Corporation  representatives,  in 
general  terms,  how  each  satellite’s  normal  and  Weibull  parameters  were 
calculated.  However,  we  did  not  analyze  any  of  the  data  used  to  calculate 
these  parameters. 


^GAO-09-325  and  GAO-1 0-636. 
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Using  the  reliability  curves  for  each  of  the  57  GPS  satellites,  we 
developed  a  Monte  Carlo  simulation  to  predict  the  probability  that  at  least 
a  given  number  of  satellites  would  be  operational  as  a  function  of  time, 
based  on  the  GPS  launch  schedule  as  of  December  2014.  We  conducted 
several  runs  of  our  simulation — each  run  consisting  of  10,000  trials — and 
generated  “sawtoothed”  curves  depicting  the  probability  that  at  least  24 
satellites  would  still  be  operational  as  a  function  of  time.  We  then  used 
our  Monte  Carlo  simulation  model  to  examine  the  effect  of  a  delay  to  the 
delivery  of  OCX  block  1  until  November  2020,  which  would  delay  the 
introduction  of  GPS  III  satellites  into  the  operational  constellation.  We 
then  reran  the  model  based  on  this  assumption,  and  calculated  new 
probabilities  that  at  least  24  satellites  would  still  be  operational  as  a 
function  of  time.  Finally,  we  simulated  the  effect  of  the  Air  Force’s 
proposed  contingency  plan,  which  would  enable  GPS  III  satellites  to  be 
added  to  the  operational  constellation,  with  limited  functionality,  prior  to 
the  delivery  of  OCX  block  1 . 

We  conducted  this  performance  audit  from  August  2014  to  September 
2015  in  accordance  with  generally  accepted  government  auditing 
standards.  Those  standards  require  that  we  plan  and  perform  the  audit  to 
obtain  sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for 
our  findings  and  conclusions  based  on  our  audit  objectives.  We  believe 
that  the  evidence  obtained  provides  a  reasonable  basis  for  our  findings 
and  conclusions  based  on  our  audit  objectives. 
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ASSISTANT  SECRETARY  OF  DEFENSE 
3600  DEFENSE  PENTAGON 
WASHINGTON.  DC  20301-3600 

AUG  28  2Ct5 

ACQUISITION 

Ms.  Christina  Chaplain 
Director 

Acquisition  and  Sourcing  Management 
U.S.  Government  Accountability  Office 
441  G  Street.  N.W. 

Washington.  DC  20548 

Dear  Ms.  Chaplain: 

This  is  the  Department  of  Defense  (DoD)  response  to  the  Government  Accountability 
Office  (GAO)  Draft  Report.  GAO-15-657,  "GPS:  Actions  Needed  to  Address  Ground  System 
Development  Problems  and  User  Equipment  Production  Readiness"  dated  July  17,  2015  (GAO 
Code  121241).  Detailed  comments  on  the  report  recommendations  are  enclosed. 


Sincerely, 


Enclosure: 
As  stated 
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GAO  Dnift  Report  Dated  July  17,  2015 
GAO-15-657  (GAO  CODE  121241) 

GPS:  ACTIONS  NEEDED  TO  ADDRESS  GROI  N'D  SYSTEM  DEVELOPMENT 
PROBLEMS  AND  I'SER  EQl  IPMENT  PRODVCTION  READINESS” 

DEPARTMEN  r  OE  DEFENSE  COMMENTS 
TO  THE  GAO  RECOMMENDATION 


RFXOMMENDA'riON  1 :  To  better  position  the  DoD  as  it  continues  pursuing  GPS 
modernization,  to  have  the  infonnation  necessary  to  make  decisions  on  how  best  to  improve  that 
modernization,  and  to  mitigate  risk-s  to  sustaining  the  GPS  constellation.  GAO  reeommends  that 
the  Secretary  of  Defense  convene  an  independent  task  force  comprising  experts  from  other 
military  serviees  and  defense  ageneies  with  substantial  know  ledge  and  e.xpertise  to  provide  an 
a.ssessment  to  USD  .AT&L  of  the  OCX  program  and  concrete  guidance  for  addressing  the  OCX 
program’s  underhing  problems,  particularly  including: 

•  A  detailed  engineering  assessment  of  OCX  defects  to  detennine  the  systemic  root  causes 
of  the  defects: 

•  Whether  the  contractor's  software  development  procedures  iuid  practices  match  the  levels 
described  in  the  OCX  systems  engineering  and  software  development  plans:  and 

•  Whether  the  contractor  is  capable  of  e.xccuting  the  program  as  currently  resourced  and 
structured. 

DoD  RESPONSE:  Concur.  OSD  commissioned  an  independent  program  assessment  team  to 
perform  annual  reviews  of  the  OCX  program  and  provide  its  assessment  to  AT&L.  Tlie 
assessment  team  consisted  of  e.xperts  from  the  .Air  Force  and  OSD  staff,  'llie  .Milestone  Decision 
.Authority  (MD.A)  found  the  review  beneficial  in  programmatic  reviews.  ITie  MD.A  was  able  to 
provide  guidance  and  direction  to  the  .Air  Force  to  address  and  correct  issues  identified  by  the 
independent  team.  We  believe  the  intent  of  this  recommendation  has  already  been  met  with  tlie 
establishment  of  this  team,  will  continue  to  review  the  progress  of  OCX  to  meeting  approved 
user  requirements,  and  recommend  no  furtlier  action  is  necessary'. 

REf 'OMMFIND.ATION  2;  To  belter  position  the  DoD  as  it  continues  pursuing  GPS 
modeniizalion.  to  have  the  information  necessary  to  make  decisions  on  how  best  to  improve  that 
modeniization.  and  to  mitigate  risks  to  sustaining  the  OPS  constellation.  G.AO  recommends  that 
the  Secretary  of  Defense  develop  higli  confidence  OCX  cost  and  schedule  estimates  based  on 
actual  track  record  for  productivity  and  learning  curves. 

DoD  RESPONSE:  Concur.  Both  the  OSD  Cost  .Assessment  Program  Evaluation  (C.APE)  and 
.Air  Force  Cost  .Analysis  .Agency  (.AFC.A.A)  perform  periodic  detailed  cost  and  schedule  analyses 
of  the  program.  'Ilie  .Air  Force  recently  completed  a  Service  Cost  Position  for  the  CK'X  program 
and  OSD  C.APE  is  beginning  an  Independent  Cost  Estimate  of  tlie  OCX  program  this  fall.  These 
organizations  alwas's  consider  actual  track  records  for  productivity  and  learning  curs'cs. 
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RKC'OMMEiNDATION  3:  To  better  position  the  DoD  as  it  continues  pursuing  UPS 
modemi/ation.  to  have  the  infomiation  necessary  to  make  decisions  on  how  best  to  improve  that 
modernization,  and  to  mitigate  risks  to  sustaining  the  OPS  constellation.  G.AO  recommends  that 
the  Secretary  of  Defense  direct  the  .Air  Force  to  retain  experts  from  the  independent  ta.sk  force  as 
a  management  advisoiy  team  to  assist  the  OCX  program  olTice  in  conducting  regular  sxstemie 
anahsis  of  defects  and  to  help  ensure  OCX  corrective  measures  are  implemented  successfully 
and  sustained. 

DoD  RESPO.N.SE:  Concur.  .Although  the  SECDEF  has  not  specifically  directed  it.  the  .Air 
Force  Program  Executive  Offiecr  for  Space  (.AFPEO'SP)  currently  uses  a  subset  of  the 
independent  program  assessment  team,  discussed  in  the  DoD  answer  to  Recommendation  1,  to 
provide  annual  systemic  analysis  of  defects  and  to  help  ensure  OCX  eoirective  measures  are 
implemented  successfully  and  sustained.  We  believe  the  intent  of  this  recommendation  has 
already  been  met,  will  continue  to  review  the  progress  of  OCX  to  meeting  user  requirements,  and 
recommend  no  further  action  is  neeessarx'. 


REC'OM.MEiSD.ATIO.N  4:  To  better  position  the  DoD  as  it  continues  pursuing  GPS 
modeniization.  to  have  the  information  necessary  to  niiike  decisions  on  how  best  to  improve  that 
modernization,  and  to  mitigate  risks  to  sustaining  the  GPS  constellation.  G.AO  recommends  that 
the  Secretary  of  Defease  put  in  place  a  mechanism  for  ensuring  that  the  know  ledge  gained  from 
the  OCX  assessment  is  used  to  determine  whether  fiulhcr  programmatic  changes  are  needed  to 
strengthen  oversiglit 

DoD  RESPO.NSE:  Concur.  Tlie  implementation  of  the  OSD  commissioned  independent 
program  assessment  team  includes  OSD  stalT  on  each  review.  We  believe  the  intent  of  this 
recommendation  has  already  been  met.  will  continue  to  review  the  progress  of  OCX  to  meeting 
user  requirements,  and  recommend  no  further  action  is  neees.saiy. 

RECOM.MEND.ATIO.N  5:  To  better  position  the  DoD  as  it  continues  pursuing  GPS 
modernization,  to  have  the  information  necessary  to  make  decisions  on  how  best  to  improve  that 
modeniization.  and  to  mitigate  risks  to  sustaining  the  GPS  constellation.  G.AO  recommends  tliat 
the  Secretary  of  Defense  to  allow  the  military  serv  ices  to  fully  assess  the  maturity  of  the  MGUE 
design  before  committing  test  and  procurement  resources,  incorporate  a  CDR  in  the  .Air  Force’s 
MGUE  development  elTort. 

DoD  RESPONSE:  Partially  Concur.  DoD  partially  concurs  with  the  recommendation  to 
conduct  a  CDR.  The  detailed  design  work  that  is  nonnally  approved  at  CDR  is  already  complete 
for  MGliE. 

'Ibis  approach  follows  the  approved  .Acquisition  Strategy  Document  path  to  compressing  the 
acquisition  process  to  deliver  capability  to  meet  the  validated  operational  need  and  the 
requirement  under  PI,  1 1 1-.383  to  procure  only  M  Code  capable  receivers,  if  available,  after  FY 
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17.  Contractors  will  deliver  MGUE  production-representative  test  articles  per  contract  schedule 
in  Jul  .Aug  2015  and  Dec  2015/Jan  2016.  well  in  advance  of  the  earliest  possible  dale  the 
program  could  insert  a  CDR  into  the  acquisition  baseline.  .A  CDR  would  require  MGf/E 
contractors  to  halt  current  development  elTorts  and  delay  both  scheduled  test  article  delivers’  and 
operational  assessments  by  a  minimum  of  6-12  months.  Tliis  would  delay  lead  platfomi 
integration  of  MCH.’E  Inc  I  capability  which  provides  the  highest  fidelity  assessment  of  maturity. 
Postponing  integration  activities  beyond  service  lead  platfomis  may  furtlier  delay  the  overall 
GPS  modernization  effort  and  increase  risk  to  the  warfigliter. 

•Adding  a  CDR  also  ntns  counter  to  the  .ASD(A)  MGUE  “Path  forward'’  memo  dated  1 1  Jun  15, 
which  states.  “The  expedited  fielding  of  tlie  MGL'E  Inc  1  capability  is  key  to  sustaining  the 
Department's  ability  to  operate  in  anti-access  area  denial  scenarios.’’  MGUE's  ability  to  meet 
pcrfomiancc  requirements  will  be  assessed  by  government  verification  testing  of  the  three 
contractors'  cards  and  testing  is  scheduled  for  completion  by  summer  20 1 6.  prior  to  any 
commitment  of  procurement  resources  by  the  military  sers’ices.  It  should  be  understood  tliat  the 
serx’ices  are  not  required  to  procure  MGUE  cards  for  lead  or  other  platfonns  until  the 
Operational  User  Evaluation  (OUE)  report  is  finalized  and  corrective  action  for  performance 
deficiencies  is  complete.  Tlie  .Air  Force  will  retain  responsibility  for  MGUE  card  development 
until  all  performance  deficiencies  found  during  the  OUE  are  addressed. 
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The  Government  Accountability  Office,  the  audit,  evaluation,  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and 
policies;  and  provides  analyses,  recommendations,  and  other  assistance 
to  help  Congress  make  informed  oversight,  policy,  and  funding  decisions. 
GAO’s  commitment  to  good  government  is  reflected  in  its  core  values  of 
accountability,  integrity,  and  reliability. 
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Connect  with  GAO 

Connect  with  GAO  on  Facebook,  Flickr,  Twitter,  and  YouTube. 

Subscribe  to  our  RSS  Feeds  or  E-mail  Updates. 

Listen  to  our  Podcasts  and  read  The  Watchblog. 

Visit  GAO  on  the  web  at  www.gao.gov. 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Website:  http://www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 

Congressional 

Relations 

Katherine  Siggerud,  Managing  Director,  siggerudk@gao.gov,  (202)  512- 
4400,  U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room 
7125,  Washington,  DC  20548 

Public  Affairs 

Chuck  Young,  Managing  Director,  youngc1@gao.gov,  (202)  512-4800 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7149 
Washington,  DC  20548 

Please  Print  on  Recycled  Paper. 

